JitMem stores raw agent trajectories untouched and lets a task-aware curator distill them into a briefing only when a new task arrives, turning memory learning into a single-step reward problem instead of a delayed-utility one.
You’ve built an LLM agent that solves a stream of tasks: shopping on a website, executing household commands, handling customer-support dialogs. You want it to get better as it accumulates experience. The standard move is write-time memory: after each task, an LLM inspects the trajectory and distills it into a persistent artifact (a reflection, a reusable skill, a workflow note). Later, a similarity search pulls the artifact back and pastes it into the prompt. Reflexion popularized this pattern, and recent work like ReasoningBank and SkillOS refines what gets distilled.
Two problems follow from committing early. First, once you compress a trajectory into “the lesson,” you can’t recover the details a future task might actually need. Second, the same trajectory contains many possible lessons (a state-change trick, a placement heuristic, a policy-check pattern), and which one matters depends on the future query you haven’t seen yet. Training the write-time curator is also awkward: the reward for a storage decision only arrives when some later task happens to retrieve it, so authors like SkillOS have to artificially group related tasks to manufacture a learning signal.
JitMem’s move is to stop compressing at write time. The memory bank keeps raw trajectories, full observation-action sequences, with no summarization. When a new task arrives, the pipeline runs four steps: retrieve top-k raw trajectories with BM25 over task descriptions, feed them plus the current task to a curator LLM that writes a compact task-specific briefing, prepend that briefing to a frozen executor’s prompt, and after execution use the executor itself as an LLM-as-a-Judge to decide whether the new trajectory is worth appending to the bank.
The curator is the only trainable piece. Because its briefing is consumed immediately on the same task it was written for, its reward (task success) arrives with zero delay. That collapses credit assignment to a single step, so the authors can train the curator with Group Relative Policy Optimization (GRPO) using nothing but the benchmark’s own success signal. No content-quality auxiliary reward, no task grouping.
One subtlety: at training time the memory bank is frozen (built by running the base executor on the train set once), so the curator’s reward reflects payload quality rather than which trajectories happened to land in the bank. At test time the bank grows online from an empty start.
for task in stream:
traces = bm25_retrieve(task, bank, k=3) # raw trajectories
payload = curator(task, traces) # task-conditioned briefing
trajectory, reward = executor(task, payload)
if executor_as_judge(task, trajectory): # quality gate
bank.append(trajectory) # store raw, not distilled
# Training: sample G payloads per task, run executor on each,
# use per-group advantage r_i - mean(r) to update curator via GRPO.
Evaluated on three agent benchmarks: ALFWorld, WebShop, and tau2-bench, against no-memory, ReasoningBank, MemP, and SkillOS baselines, with three frozen executors (Qwen3-8B, Gemini-2.5-Pro, GPT-5.4).
•
Trained JitMem beats the strongest baseline by +16.2, +16.3, and +3.9 success-rate points on ALFWorld, WebShop, and τ²-bench respectively. On WebShop with Qwen3-8B executor, that’s 32.8 SR vs. 16.5 for RL-trained SkillOS.
•
Even the untrained curator is competitive. With Gemini-2.5-Pro as both curator and executor, prompted JitMem hits 61.0 SR on WebShop vs. 41.0 for prompted SkillOS. The authors read this as evidence that read-time task adaptivity, not curator model strength, is the main lever.
•
The trained curator transfers. A curator trained with a Qwen3-8B executor, then used with GPT-5.4 at test time, comes within 1.4 SR points of a curator trained directly with GPT-5.4 (ALFWorld). One curator can serve multiple executors.
•
Compact context. On ALFWorld with GPT-5.4, JitMem adds far fewer input tokens than write-time baselines and cuts executor steps by roughly a third versus ReasoningBank and SkillOS-base.
•
Ablations isolate the design choices. Removing task conditioning, storing failed trajectories with labels instead of filtering, or pre-distilling at write time each degrade performance independently. Forcing the retriever to return nothing drops the trained curator by up to ~15 SR points, so RL is genuinely learning to use retrieved experience, not memorize task-solving tricks.
•
Weak spot. On τ²-bench’s Airline and Retail domains, no memory method beats the no-memory baseline beyond variance. The gains come from Telecom, where multi-step policy verification benefits from procedural guidance.
•
If you’re building a streaming-task agent and currently use write-time reflections or skills, the training-free version of this idea is cheap to try: keep raw trajectories, run BM25 over task descriptions, and add a curator prompt that takes both the current task and the retrieved traces. The paper’s evidence is that even without training, this often matches or beats write-time distillation using the same curator model. Worth testing before you invest in an RL loop.
•
If you already RL-train a write-time memory policy and struggle with delayed rewards or task-grouping heuristics, the read-time framing is the interesting structural change. The reward becomes immediate, so plain Group Relative Policy Optimization (GRPO) with the benchmark’s own success signal is enough. The authors do 100 steps at batch size 32, group size 8, on 8 H200s.
•
If you serve multiple executor models, the transfer result suggests you can train one curator against a cheap executor and reuse it with stronger ones. The paper shows this within a family of general-purpose LLMs; whether it holds across more specialized executors is worth testing.
•
Read-time curation adds an extra LLM call per task. If your bottleneck is latency rather than executor steps, the input-token savings the paper reports may not translate to wall-clock wins for you.
•
The gains concentrate on tasks that need procedural guidance (household planning, web navigation, policy-heavy tool use). For fact-retrieval-style tasks, the paper’s own τ²-bench Airline and Retail numbers suggest limited benefit.
•
The retriever is plain BM25 over task descriptions. As banks grow more diverse, the authors flag retrieval as a likely bottleneck; nothing in the paper stress-tests that.
•
The training bank is frozen and built from base-executor trajectories, while the test bank grows with curator-augmented ones. The “staged bank refresh” ablation, which rebuilds the training bank after 100 steps, adds only modest gains, so the distribution shift appears small in the tested regime but isn’t eliminated.
•
Trained JitMem results are for ALFWorld and WebShop only; τ²-bench uses training-free variants because it lacks a standard training split, so the +3.9 headline number on τ²-bench comes from a prompted curator, not a trained one.
•
The payload format is hand-designed per benchmark. Whether the same template transfers to a new domain without prompt engineering is not shown.
•
“Executor-as-judge” gating the bank means low-signal or adversarial domains could quietly poison memory. The paper doesn’t probe judge failure modes.