BCIT decides whether a past fine-tuning trick still deserves compute after the parent model has changed, by binding each recorded gain to its source context and forcing a short current-parent trial when evidence is stale. Under matched compute it authorizes 25% of harmful candidates versus 62.5% for an additive-score baseline.
You run an autonomous Post-training loop for a 4B model your team ships. The loop proposes updates (a new SFT mix, a Group Relative Policy Optimization (GRPO) recipe, a data filter), trains them, evaluates, and logs what worked. After a few rounds you have a library of “this update gained 2 pp on SQL last month” records. The tempting move is to replay the winners on the current checkpoint. But the current checkpoint isn’t last month’s checkpoint, so a past win can now silently wreck instruction following or degrade an unrelated capability.
Most prior autonomous-training work (LaMDAgent and similar pipeline agents) focuses on proposing and training candidates well. It treats past outcomes as roughly context-free hints. This paper argues the missing piece is a gatekeeper that decides, per candidate, whether old evidence is still trustworthy enough to spend a full training run on.
Every historical update is stored with the parent it was trained from, the data and eval protocol it used, the size of the observed gain, and a set of pre-declared applicability conditions (which capability family, which training regime, which output interface, etc.). When a new decision comes up, BCIT scores two things: source strength (how big and how well-replicated was the original gain) and current compatibility (how many applicability conditions still hold against today’s parent). It multiplies them rather than averaging, so a great past result cannot paper over a context mismatch.
On top of that sits a hard-conflict veto: if a named required condition is explicitly contradicted (say, the update assumed a different output format than today’s protocol), the candidate is rejected outright, regardless of score. Everything else routes into three buckets: Reject, Validate (a small budget-capped training run from the current parent, whose checkpoint is thrown away), or Train (full budget). Only the top grade of evidence, replicated matched-control comparisons, can skip validation. New agent-generated proposals never skip it, because they have no observed effect yet.
Crucially, authorization is separate from promotion. Every fully trained child, no matter which policy authorized it, faces the same shared adoption rule on held-out data before it can replace the parent.
for candidate in stream:
if not executable(candidate): continue
if candidate.hard_conflict: reject(); continue
S = source_strength(candidate) # discounted by evidence grade
A = current_compatibility(candidate) # mean over applicability fields
Q = S * A
if Q >= 0.70 and candidate.grade == 'A': full_train(candidate)
elif Q < 0.30: reject()
elif faithful_short_run_exists(candidate): validate(candidate)
else: reject()
# every trained child then faces the shared promote-or-rollback rule
The prevailing habit in autonomous post-training is to treat a logged win as a green light: it worked before, run it again. This paper says the opposite. An observed gain is evidence bound to the parent, data, and protocol under which it happened. It is not a permission slip to spend full training compute from a different parent. The cleanest evidence is the outcome-blind authorization audit on 24 candidates, where removing the applicability check alone pushes harmful authorization from 25% up to 75%.
The load-bearing finding is that update effects are genuinely unstable across contexts: across 24 matched candidate-context pairs, only 11 improved their target metric, and only 3 of those 11 also held both retention measures. A standard SQL SFT update gained 2.25 pp on BIRD while destroying ~22 pp of IFEval prompt-level accuracy. That instability is what makes the gatekeeping problem real rather than academic.
Given that, the authorization audit shows:
•
BCIT authorizes 2 of 8 harmful candidates and 9 of 10 beneficial ones. Flat-additive scoring authorizes 5 of 8 harmful and 8 of 10 beneficial.
•
Removing the applicability check alone: harmful authorization jumps to 75%. Removing the hard veto: 50%. Both components carry weight.
•
The paired McNemar test on harmful decisions gives p = .25: directional cohort evidence, not a population claim. The authors are explicit about this.
On bounded validation as a signal: short-run and full-run target directions agree on 20 of 24 candidates at a 20% validation budget, with two false passes remaining. Useful but not a substitute for full training.
Under a matched 36-GPU-hour budget adapting one Qwen3-4B across TAT-QA, BIRD, and Berkeley Function Calling Leaderboard (with IFEval as retention guardrail), BCIT reaches a cross-task mean of 47.0 versus 44.4 for flat-additive, 45.5 for validate-everything, and 46.1 for additive-plus-veto. The paired gain over flat-additive is +2.63 pp and positive in all six seeds. Against the stronger validate-all baseline BCIT wins by +1.50 pp while using 0.74 fewer GPU-hours, because it skips validation on candidates the gate already trusts or rejects.
Reach for this when you’re running an automated fine-tuning loop that keeps a library of past successful recipes. Today you probably retrieve top-scoring past updates and re-run them on the current checkpoint. BCIT’s pattern says: attach each stored recipe to the conditions it was measured under (parent stage, data mix, output protocol, retention constraints), score those conditions against today’s setup, veto on any named contradiction, and force a cheap current-parent trial when the score is ambiguous. The score formula (source strength × current compatibility, not sum) is the important detail; a sum lets a great old result override a bad context match.
The paper releases no code or artifact link. Reimplementing the mechanism is straightforward: the thresholds (0.30 low, 0.70 high, evidence-grade discounts of 1.0/0.75/0.5) and the applicability fields are fully specified in the appendix. Datasets used are all public (FinQA, Spider, xLAM as sources; TAT-QA, BIRD, Berkeley Function Calling Leaderboard as targets; IFEval as retention).
A logged win is evidence about the parent that produced it, not a license to spend compute from a different parent. This matters most in loops where each promotion silently changes the baseline for every future decision, so context-free reuse compounds into trajectory drift. The cheap fix is a multiplicative gate plus a named-conflict veto, not a smarter retrieval score.
•
Everything is one 4B model, three target capabilities, one retention benchmark, and six paired seeds. The headline authorization comparison is directional (McNemar p = .25), not a population error-rate claim; the authors say so.
•
The applicability conditions and hard conflicts are human-specified per candidate. If your team can’t or won’t write those fields honestly at logging time, BCIT degrades toward the additive baseline it beats. Learned boundaries are future work.
•
Bounded validation still produces two false passes in 24 candidates, and the fallback that lets inconclusive results proceed to full training is a study setting rather than a proven default. Weaker validation budgets (5-10%) agree with full runs noticeably less often.