BDH-CQ solves visual reasoning puzzles by iterating on a continuous hidden state instead of emitting a chain-of-thought, hitting 29.5% pass@2 on ARC-AGI-1 at $0.00070 per task, roughly one-tenth of a cent and past the public cost-accuracy frontier.
You’ve shipped an agent that solves novel per-user tasks from a few examples in the prompt. Today it burns thousands of reasoning tokens per call, most of which are the model narrating to itself before producing the actual answer. That serial narration is where your latency and cost live.
The dominant baseline is Chain-of-Thought prompting on top of a big autoregressive model: demonstrations tell the model what to do, generated intermediate tokens do the work. BDH-CQ keeps the demonstrations-in-context part but replaces the token scratchpad with repeated updates to a continuous internal state. Only the final answer is decoded. The claim isn’t a new SOTA accuracy on ARC-AGI-1, it’s that a 150M-parameter model with this design lands on the cost-accuracy frontier of a benchmark usually dominated by frontier LLMs and by task-specific solvers like Hierarchical Reasoning Model (HRM) and Tiny Recursive Model (TRM) that cost dollars per task.
The system keeps two separate pieces of state. One, called S, is a recurrent memory that gets updated as each demonstration example streams in. Think of it as a running summary of “what transformation is this task asking for,” written by a fixed network, no gradient updates at inference. This plays the role attention plays in a Transformer, but without a growing key-value cache. The paper connects it to Fast-weight memory and Linear attention as prior formalisms for this kind of contextual association.
The second piece, H, is a reasoning workspace that only comes alive after all demonstrations plus the query have been ingested. The model then iterates: take H, take the final memory S, produce a new H, repeat for some number of steps, decode the answer. No tokens are emitted between iterations. This is what the authors mean by recurrent latent reasoning, and it’s cousin to COCONUT (feed the last hidden state back as input) and to Recurrent-depth models models that apply the same block many times.
Crucially, unlike Hierarchical Reasoning Model (HRM) or Tiny Recursive Model (TRM), no per-task optimization happens at inference. The 150M model is trained once on an ARC-style mixture (RE-ARC, ConceptARC, ARC-Heavy, ARC-GEN100K, plus private curation) and then frozen. Exact dimensions and update rules are marked proprietary in the paper.
# Fixed params theta; no updates at inference.
S = init_memory()
for (x, y) in demonstrations:
S = U_theta(S, encode(x, y)) # recurrent context
H = E_theta(query_input, S) # seed workspace
for r in range(R): # R latent reasoning steps
H = F_theta(H, S) # no tokens emitted
answer = G_theta(H) # decode only the final grid
The architecture descends from BDH, the Dragon Hatchling (BDH) sequence model the same group published earlier.
The prevailing move for hard reasoning is to spend more test-time compute as more generated tokens: longer chains of thought, self-verification, tree search over verbalized steps. This paper argues you can spend that compute inside the model instead. Iterate a continuous state, keep several candidate hypotheses live in one vector, and decode only when you have an answer. The load-bearing evidence isn’t the headline score, it’s the controlled experiments showing that a frozen 150M-parameter model actually binds new operators from context and extrapolates them, at a cost point that verbalized systems can’t reach.
The headline: 29.5% pass@2 on the 400-task public ARC-AGI-1 eval at a computed $0.00070 per task, using about 0.85 H200-GPU-seconds. The paper positions this as roughly 57x cheaper than GPT 5.6 Luna (Low) at 34.2% by the leaderboard’s own numbers, or ~11x cheaper after a subsequent OpenAI price cut. A separate black-box audit reproduced the 29.5% number without model weights.
The behavioral experiments are where the mechanism gets tested:
•
Dense contextual binding works cleanly. Given demonstrations that define a fresh color permutation, the model applies it correctly on 96/96 held-out outputs even as the number of simultaneous color bindings grows from two to eight.
•
Simple operators extrapolate. Extending a shape to a boundary and copying a motif to marked anchors: 48/48 correct across the tested ranges, even at distances and multiplicities beyond what the demonstrations showed.
•
Sequence-length and nesting expose ceilings. Ordering bars by length is near-perfect through five objects, then collapses to 1/24 at length eight. Nested-containment recoloring is near-perfect through depth four, then falls to 29/36 at depth five.
•
Coverage in the prompt fixes some ceilings, not others. Adding one demonstration at the target complexity lifts depth-five nesting from 19/24 to 24/24, so nesting failure was mostly failure to extrapolate. The same intervention only partially rescues length-eight ordering (0/24 to 13/24), suggesting a real execution bottleneck there.
•
Composition is representation-dependent. Rotation-then-relocation: 72/72. Reflection-then-relocation: 47/72. A color-swap operation basically fails to compose (0/72). So “can this model compose two learned operations” has no single answer, it depends on the operations.
•
Within-task consistency is imperfect. On ConceptARC, 52 of 160 tasks have one or two correct test inputs but aren’t solved as whole tasks under strict grading, meaning the model sometimes produces the right output without having stably inferred the rule.
•
Reasoning effort is a dial. Training with variable latent-iteration counts lets you pick LOW/MEDIUM/HIGH at inference; more iterations trade cost for pass@2 monotonically.
Reach for this line of work when you’re building a system that has to learn a fresh per-request task from a handful of examples and then execute it many times, and where you can verify the output exactly (grid match, unit test, constraint check). The BDH-CQ recipe suggests: don’t spend inference budget on the model narrating its plan, spend it on more latent iterations of a fixed network with the examples already written into its memory. The reasoning-effort knob lets you sweep the cost-accuracy curve at deploy time without retraining.
On artifacts: the paper releases essentially nothing runnable. Architecture dimensions, update rules, training recipe, and the exact data mixture are marked proprietary. There’s no code repo, no weights, no reproduction script, only a black-box audit by external co-authors. Practically, you’re reading this for the design pattern (context-updates-memory, latent iteration decodes answer) and for the behavioral map of what such a system can and can’t do, not to pick up an implementation.
When your task is verifiable and repeats a learned operation many times per request, spend compute on latent iteration instead of on tokens the model reads back to itself. The catch is that you need training data structured as (demonstrations, query, exact answer) triples, and you need to accept that debugging is harder because there’s no reasoning trace to inspect, only the final output.
•
Closed system. Dimensions, update rules, training recipe, and much of the data mixture are proprietary. You can’t reproduce or ablate the architecture from the paper; you can only take the design idea and re-derive it.
•
One benchmark family. All results are on ARC and ARC-like grids. The claim that this generalizes to language, math, or agent tasks is an outlook section, not evidence. Composition results already show the model’s behavior is representation-dependent in ways the paper doesn’t fully explain.
•
Cost comparison depends on leaderboard accounting. The 57x-cheaper framing uses ARC Prize’s reported costs for other systems, which mix API prices and hardware estimates; the 11x number after OpenAI’s price cut is the same comparison recomputed. Nothing in the paper compares BDH-CQ against a same-size Transformer trained on the same ARC mixture, so we don’t know how much of the win is the latent-reasoning design versus the data curation and the pass@2 candidate ranking pipeline.