Compile by training turns a natural-language function spec into a small local neural function by having teacher LLMs synthesize examples that finetune a LoRA adapter on a shared 0.6B interpreter, lifting semantic accuracy from 22.4% to 83.6% on tasks where the fast one-shot compiler produced no exact matches.
You’re building a product that needs to classify support emails, extract order IDs from messy text, or route user questions to the right handler. Today you either write brittle regex, or you hit GPT-5 on every request and eat the latency, cost, and vendor lock-in. Neither fits: the task is too fuzzy for rules, too repetitive to justify a frontier call per invocation. This paper’s answer is to treat the frontier model as a compiler, not a runtime. You describe the function in English once, pay a one-minute build cost, and ship a small local artifact you can version and cache like any other binary. It extends Program-as-Weights (PAW), which did the same thing but predicted the adapter weights in a single forward pass. Fast, but capped in quality.
The system exposes two calls: Compile(spec) returns a program, Run(program, input) returns an output. Every compiled program shares one frozen Qwen3-0.6B interpreter; what differs per function is a small LoRA adapter plus a prompt scaffold. Compilation has two stages. First, teacher models (a mix of GPT-5.4-mini and GPT-5.5) generate a task-specific dataset from the spec, returning structured JSON pairs that get validated and rejected if malformed. Second, those pairs finetune the adapter using standard next-token cross-entropy, warm-started from PAW’s amortized one-shot prediction rather than random init. The finished artifact packages adapter, scaffold, spec, and interpreter metadata into a .paw file. To keep the minute-scale build feel interactive, teacher synthesis and training run concurrently: training kicks off as soon as its first batch is filled and blocks only if it catches synthesis. A job queue dispatches to shared GPU workers and caches teacher outputs across jobs.
def compile_by_training(spec):
scaffold, theta0 = paw_amortized(spec) # seconds, warm start
dataset = []
for batch in stream_teacher_examples(spec): # overlap I/O
dataset.extend(validate_json(batch))
if len(dataset) >= batch_size:
theta0 = sgd_step(theta0, scaffold, dataset)
return package(theta0, scaffold, spec)
The prevailing move when you want a cheap local model for a fuzzy task is either (a) prompt a big model at runtime forever, or (b) do a one-shot amortized weight prediction like Program-as-Weights (PAW) and accept whatever quality falls out. This paper argues the right framing is compilation with a real optimizer in the loop: spend a minute of GPU time and teacher tokens once, get a versioned local artifact whose quality is much closer to the teacher’s. The evidence is direct: on the exact task subset where the one-shot compiler produced zero exact matches, running gradient descent on teacher-synthesized examples closes most of the gap.
On FuzzyBench-Hard, compile-by-training reaches 0.836 mean LLM Exact Match (LEM) versus 0.224 for PAW’s fast amortized compiler, an absolute jump of +0.612. This is the load-bearing result: the hard subset was defined as specs where the fast path produced no exact matches, so the lift shows the extra optimization is doing real work rather than polishing already-solved cases.
•
Teacher mix matters. Mixing GPT-5.4-mini with GPT-5.5 supervision at 2:1 raises mean LEM from 0.746 to 0.851 versus mini alone, at the same total example count.
•
Data scaling is soft. Going from 1440 to 7200 unique pairs moves LEM from 0.821 to 0.866, with a plateau at 2400–3600 pairs.
•
Interactive latency holds. Cold compile of a representative spec: 50.9s on a B300, 68.2s on H200, 99.2s on RTX. Four concurrent jobs finish with ~1s mean queue wait.
•
Composition works. Paw-helper runs 28 compiled functions live behind one router across four websites. An avatar controller compiled the same way produced the expected action DSL on 43 of 44 hand-written instructions. An English–Claudish translator served 100,747 requests in ~11 days.
Reach for this when you have a recurring text-to-text task in your product (email triage, ID extraction, intent routing, style rewriting) that you’d otherwise handle with a GPT-5 call per request. Write the spec in English, let the service pay the one-time teacher cost, download the .paw artifact, and run it locally through the SDK behind whatever deterministic glue code (retrieval, caching, validation) your app already has. The website-helper pattern is the template: compiled functions make fuzzy calls, ordinary code handles exact operations.
Artifacts: the compiler runs at programasweights.com/playground. The website-helper reference implementation is on GitHub, and the Claudish translator specs and code are also public. The paper does not release FuzzyBench-Hard itself or the training-data recipe as a downloadable dataset; it’s a hosted service.
Treat frontier LLMs as compilers, not runtimes: spend a minute of teacher tokens once to bake a versioned local function, instead of paying for a remote call on every input. The trick is having a shared frozen interpreter so the per-function artifact is tiny enough to store, cache, and compose like ordinary software.
•
Correctness is bounded by the teacher. Anywhere GPT-5.5 is wrong or biased, the compiled function inherits it silently. The authors flag this and recommend deterministic validation for anything safety-critical.
•
Quality is measured by an LLM judge (LLM Exact Match (LEM)), not human eval or exact match. The judge scores 0.977 against 128 author labels, which is strong but still one grader on one benchmark subset.
•
The application evidence (website helper, avatar, translator) is deployment demos and one 44-item validation set, not controlled user studies. How well this composes at 100+ functions with real end-user traffic is unmeasured.