InterOPT tackles a specific failure of LLM optimization assistants: they start building math models before the user has actually said what they want. It maintains a running ledger of unresolved business gaps and forces each question to bind to one, cutting silent assumptions from 0.69 to 0.37 per run in the multiple-choice setting.
You’ve shipped an LLM assistant that turns customer requests into runnable code. A user says “schedule my delivery vans across 30 stops, minimize cost.” Your agent produces a formulation. But the user never said whether vans return to the depot, whether time windows are hard, or whether unmet demand can be backlogged. Each of those changes the math. Today most agents either silently pick a default or ask a vague “anything else?” and move on.
Prior OR benchmarks like NL4Opt and OptiMUS assume the user handed over a complete textbook problem, so they never measured whether an agent knows it’s missing information. One interactive system, ORPilot, does interview users, but its clarification quality was never isolated as its own metric. This paper argues the pre-modeling clarification step deserves its own benchmark and its own method.
The authors first define a formulation-critical fact: one whose value would change the objective, constraints, or decision variables of the resulting math program. Their benchmark OR-Clarify takes 100 fully specified OR problems, decomposes each into atomic facts, hides roughly half of the modeling-relevant ones (178 hidden slots total, tagged P0/P1/P2 by severity), and gives the agent only the partial brief. A simulated user answers questions but only from the private facts, never volunteering anything unasked. A post-hoc judge scores whether each hidden slot was exactly recovered before the agent said READY_TO_MODEL.
InterOPT itself splits clarification into two stages that most LLM agents conflate. Stage 1, Dynamic Gap Search, runs a structured LLM call each turn that scans the public transcript for missing conditions across six categories (objectives, decision scope, constraints, time boundaries, entity relationships, hard-vs-soft policies) and appends up to three new gaps to a persistent ledger. Stage 2, Gap-Guided Action Search, generates three candidate questions, each bound to a specific open ledger entry, then a selector picks one. The stop-or-ask decision stays with the model, not the ledger. In the Choice protocol variant, each question ships with three answer options.
ledger = []
for t in range(T_max):
ledger = update_gaps(brief, transcript, ledger) # Stage 1
open_gaps = [g for g in ledger if g.status == "Open"]
decision, states, questions = propose(brief, transcript, open_gaps) # Stage 2
if decision == "READY_TO_MODEL":
return transcript
# each question must cite an open_gap id when any exist
action = select(transcript, states, questions, open_gaps)
answer = simulated_user.reply(action)
transcript.append((action, answer))
mark_bound_gap_as_asked(action)
The usual instinct when an LLM agent looks underinformed is to prompt it to “ask better questions.” This paper shows the opposite. The bottleneck isn’t question phrasing; it’s that the agent has no persistent memory of what it doesn’t yet know, so it keeps re-deciding readiness from scratch each turn. The clearest evidence is the ablation: dropping Stage 1 (the ledger) hurts Core Exact more than dropping Stage 2 (the guided asker) under the Choice protocol, an 11.1 vs 6.0 percentage-point drop.
Under the constrained Choice/MC-D protocol, InterOPT reaches 0.675 Core Exact vs 0.506 for the plain MC-D baseline and 0.517 for a version with just an added stopping reviewer. Silent assumptions per run fall from 0.692 to 0.366. The mechanism check: of 890 judged slots, InterOPT gets 617 full recoveries vs 461 for MC-D, with almost no change in partial credits (20 vs 16). So the gain comes from converting outright misses into full hits, not from marginal near-misses.
Secondary findings worth knowing:
•
In the open free-form setting, InterOPT does not win. The ORPilot adapter reaches 0.583 Core Exact vs InterOPT’s 0.538. The authors are upfront that gap-guided clarification doesn’t uniformly dominate free-form baselines.
•
Cost is real. InterOPT under Choice uses 10.7 turns and 10.0 atomic questions on average, roughly 3x the baselines. The paper frames this as a coverage-cost trade-off, not a free lunch.
•
Across four off-the-shelf models (DeepSeek-V4-Pro, GLM-5.1, GPT-5.5, Claude Opus 4.8) with no InterOPT scaffolding, no model exceeds 60% Core Exact on OR-Clarify, and all leave substantial silent assumptions. Opus-4.8 is strongest but still leaves 0.42 silent assumptions per run.
•
In open-mode diagnostics, 40.6% of InterOPT runs are flagged as premature stopping. The failure isn’t asking badly, it’s not knowing when to keep asking.
Reach for this pattern when you’re shipping any agent that turns a fuzzy business request into a structured artifact (SQL, an optimization model, an API call sequence, a workflow config) and where getting the spec wrong is expensive. The generalizable move is not the OR-specific gap categories, it’s the two-stage split: keep a persistent ledger of “things I noticed I don’t know” separate from the “what should I say next” call. Bind each user-facing question to a specific ledger entry so you can audit later why you asked.
The paper doesn’t mention a code or data release, and no repository URL appears in the text provided. If you want to replicate, the benchmark construction recipe (decompose facts, screen for formulation-relevance, mask ~50%, annotate severity) is described in enough detail to rebuild for your own domain. The Choice protocol with a fixed “none of the above + free-form correction” option (MC-D) is the cheapest useful pattern here and worth stealing on its own.
Give your agent a place to write down what it doesn’t know, or it will keep pretending it knows enough. The ledger is doing more work than the clever asker on top of it, and this matters most when the downstream artifact is structured enough that a silent default becomes a silent bug.
•
InterOPT wins clearly only in the Choice protocol. Under free-form clarification it loses to ORPilot on the same benchmark, so the ledger-plus-selector design may be exploiting the discrete answer space more than genuine gap reasoning.
•
The interaction budget roughly triples. If each user turn is a real human, 10 clarification questions before you even start modeling is likely unshippable and the paper doesn’t study user tolerance.
•
The benchmark is 100 cases with a simulated user built from the same fact set the judge scores against. Real users volunteer, contradict, and change their minds, none of which OR-Clarify tests.