MAPD distills proprietary LLMs into open-source agentic search models by routing supervision through a style-normalized JSON protocol instead of raw teacher text, keeping the student’s own token distribution intact while a same-model privileged branch supplies dense per-token guidance.
You’ve shipped a small open-source retrieval agent (say, a Qwen-class 1.7B or 4B model) that plans a query, calls a search tool a few times, and answers. You want it to reason more like Claude or GPT does on hard multi-hop questions, but you can’t touch their logits and their tokenizers don’t match yours. The obvious fallback, fine-tuning on transcripts of the big model’s chain-of-thought, is known to backfire: the student picks up the teacher’s verbose phrasing and confident tone without the underlying skill, a failure mode often called false promise imitation. Meanwhile, the standard training recipe for these agents, outcome-only RL with verifiable rewards, only tells the model “final answer right/wrong” across a 20-step trajectory, which is very thin feedback.
MAPD has two stages. Offline, a proprietary model (Claude-Opus-4.6, GPT-5.5, or Gemini-3.1-Pro in the paper) drives a small Multi-Agent System: an Orchestrator decomposes the question into sub-tasks with dependencies, parallel Searchers hit a local Wikipedia index, a Repair agent diagnoses failures using the gold answer as a hint (the gold answer is never written into the trace itself), and a Protocolizer compresses everything into a JSON object with five fields: task type, reasoning plan, extractive grounding facts, partial findings, and a verified answer. A quality gate checks schema, exact-match on the answer, that every grounding fact is a verbatim substring of a retrieved passage, and that no oracle info leaked into the plan. Only ~99% clean protocols survive.
Online, the trick is On-Policy Self-Distillation. The same student weights are run twice per token: a student branch sees only the question and its own prior tokens, a privileged branch also sees the JSON protocol. The student is trained to match the privileged branch’s next-token distribution via reverse KL, and because both branches share weights and tokenizer, there is no cross-vocabulary alignment problem. This dense loss is added to a standard Group Relative Policy Optimization (GRPO) outcome-reward objective, weighted by a coefficient the paper calls lambda.
for question, gold in train_batch:
protocol = cached_mas_protocol[question] # offline JSON
rollouts = student.sample(question, n=8) # for GRPO
for t in tokens(rollouts):
p_student = student(question, prefix_up_to(t))
p_priv = student(question, protocol, prefix_up_to(t))
loss_opsd = KL(p_student || stop_grad(p_priv))
loss_rl = grpo(rollouts, reward=exact_match(gold))
update(student, loss_rl + 0.05 * loss_opsd)
The common move when distilling a black-box teacher into an open student is to fine-tune on the teacher’s natural-language reasoning traces. This paper argues the opposite. The teacher’s surface language is the part you must throw away; only the abstract plan and the grounded facts should cross the gap, and they should cross in a schema the student never has to imitate at inference. The load-bearing evidence is the ablation where distilling proprietary raw text actually underperforms using no proprietary teacher at all.
•
The ablation that carries the thesis: distilling from a proprietary model’s raw natural-language trajectory (w/o SP&MAS) is worse than a baseline that uses no proprietary teacher (GRPO+OPSD on self-rollouts). Swapping raw text for the JSON protocol under a single-teacher setup adds +7.0 points on 1.7B and +5.6 on 4B. Structure, not teacher strength, is what unlocks the transfer.
•
Headline: on seven QA benchmarks (NQ, TriviaQA, PopQA, HotpotQA, 2WikiMultihopQA, MuSiQue, Bamboogle), MAPD averages 39.4% on Qwen3-1.7B and 44.4% on Qwen3-4B, beating the strongest prior hybrid SDAR by 4.8% and 3.3% relative.
•
Multi-hop gains are roughly 3x the single-hop gains (7.9% vs 2.3% relative on 1.7B), which lines up with what a decomposition-heavy MAS should transfer.
•
Pure On-Policy Self-Distillation without an external protocol collapses: 5.9% on 1.7B, with response-length clipping jumping from 5% to ~74% as the model degenerates into runaway outputs.
•
The distillation weight has a real sweet spot at lambda=0.05. Push it to 0.1 and the 4B model shortcuts to “retrieve without reasoning”: mean response drops from 135 to ~42 tokens, tool calls saturate at 3 per episode, single-hop scores tick up but multi-hop scores fall sharply (2WikiMultihopQA 45.7% -> 38.4%).
•
Teacher swap is nearly free: Claude, GPT, and Gemini as the offline teacher land within 2 points of each other on both student sizes. Offline synthesis cost was about $1,454 one-time for 25.6K training instances with Gemini.
Reach for this when you’re training a small open-source agent (retrieval, code search, tool-using QA) and you already pay for a frontier API but can’t get logits out of it. Instead of SFT on the frontier model’s transcripts, run it in an offline planner/searcher/repair loop, dump each solved episode to a strict JSON schema (plan + extracted evidence + verified answer), and use that JSON only as privileged context in a same-model self-distillation branch alongside your existing RL loop. Your production agent never sees the JSON at inference, so there’s no serving overhead or format lock-in, and your dispatcher can swap teacher APIs without retraining.
Code is released at GitHub. The training data is the standard NQ + HotpotQA splits from Search-R1, retrieval runs against the wiki-18 corpus, and the students are Qwen3-1.7B and Qwen3-4B base checkpoints. The paper does not release the pre-synthesized protocol cache, so you would regenerate it yourself.
When you can’t touch a teacher’s logits, distill its plan, not its prose. A rigid JSON schema for “what to do and what facts support it” transfers the useful part of a frontier model’s reasoning without dragging its writing style into your smaller model’s distribution, and it lets you keep the teacher entirely offline.
•
The whole approach depends on there being a verifier (exact-match against a gold answer) strong enough to drive both the offline Repair agent and the online RL reward. Open-ended tasks without an EM-style check don’t obviously fit.
•
The lambda=0.1 failure is a warning: too much protocol pressure teaches the model to skip reasoning and just retrieve. The good operating point is narrow and had to be tuned per model scale.
•
Evaluation is Wikipedia QA at 4 turns and 512-token responses with a top-3 retriever. Longer-horizon agents (code, browsing, multi-tool) may need a richer protocol schema than the five fields used here, and the paper doesn’t test that.