Enoki detects LLM hallucinations at both the claim level and the span level from a single intermediate representation, text-anchored Open Information Extraction triples that get verified and then projected back to the exact answer characters, so a rule-based backend can match LLM-heavy pipelines at ~100× lower latency.
You’ve shipped a RAG-based support assistant. A user complains that one paragraph in a long answer is wrong. Today you have two bad options: run a claim-decomposition pipeline (many LLM calls, gives you a list of unsupported statements but not where in the answer they live) or run a span detector (highlights bad text but can’t tell you which fact was wrong). Bridging these usually means a third stage that aligns unsupported claims back to character offsets, and errors compound at each step. Existing systems pick a lane: FActScore and similar tools verify decomposed claims; LettuceDetect and RAGTruth-tuned encoders localize spans. Enoki’s pitch is that one representation can do both jobs without an alignment module.
The core move is to never let the extracted facts drift away from the original text. When Enoki decomposes an answer sentence into (subject, predicate, object) triples, it keeps the hallucination-relevant arguments anchored to their exact character spans in the answer. Verification is standard Natural Language Inference: turn each triple into a hypothesis, score it against the evidence, call it hallucinated if entailment probability is low. The payoff comes at projection time. If a triple fails, you already know which characters produced its object argument, so you emit those characters as the hallucinated span. No separate claim-to-span matcher.
Two details make this work in practice. First, incremental decomposition: Enoki extracts nested triples of increasing specificity from one sentence (coarse fact, then refinement adding a modifier, then refinement adding another). If the coarse fact is supported but a refinement isn’t, only the newly-added delta span gets flagged. Second, three interchangeable extractors share the verification interface: Enoki-LLM (a CycleOIE-style prompt extended with incrementality guidelines), Enoki-Encoder (a ModernBERT-large trained with Iterative Grid Labeling but supervised via Hungarian matching so row order doesn’t matter), and Enoki-Rule (35 hand-curated spaCy dependency patterns).
for sent in spacy_sentences(answer):
triples = extractor(sent) # LLM, encoder, or rule
for group in incremental_groups(triples):
for triple in group: # narrow -> wide
hyp = to_hypothesis(triple)
score = max(nli(hyp, chunk) for chunk in evidence_chunks)
if not_entailed(score):
yield span_of_delta(triple, group) # char offsets
The rule library itself was grown by a coding agent under a fixed acceptance gate: propose a rule, measure F1 plus a coverage bonus on held-out extractions, commit only if the score strictly improves on two random seeds.
The usual assumption is that claim-level verification and span-level localization are different problems that need to be stitched together with an alignment step. Enoki argues the opposite. If you refuse to let the extracted facts detach from their source characters in the first place, verification and localization become the same operation with two different views on the output. The clearest evidence is that a purely rule-based extractor plus a small NLI verifier stays close to LLM pipelines on span metrics while running ~100× faster, which only makes sense if the representation, not the extractor’s cleverness, is doing the work.
The load-bearing result is that the extractor can be swapped down to a rule-based system without collapsing performance. Enoki-Rule and Enoki-Encoder both beat standard OpenIE baselines on span localization and stay competitive with LLM-heavy detectors, at 0.11s and 0.13s per sentence vs 11.95s for Claimify. That’s the two-orders-of-magnitude gap that makes the shared-representation claim credible.
•
On HalluEntity (entity-level), Enoki-LLM improves over the strongest prior detector by +15.3 AUPRC.
•
On Mu-SHROOM (span-level), +8.0 Span Coverage F1 over the strongest baseline.
•
Sentence-level, Enoki-Encoder hits 69.1% F1 at 0.13s, 4–10× faster than baselines at comparable accuracy; Enoki-LLM tops the list at 76.4% F1, beating Claimify by +9.8 points.
•
Swapping decomposition from open-source GPT-OSS-120B to GPT-5.4 only moves HalluEntity by +1.85 AUROC / +1.90 AUPRC, suggesting the LLM backend isn’t the bottleneck.
•
The optional coreference preprocessing helps on RAGTruth and ANAH (where cross-sentence references matter) and hurts slightly elsewhere, so it’s kept off by default.
They also release EnokiQA: 3,990 labeled and 19,594 unlabeled long-form QA examples with aligned claim- and span-level annotations, longer answers and evidence than prior fine-grained datasets.
Reach for this when you’re shipping a RAG-grounded assistant and want a hallucination guard that both flags which fact is wrong and highlights the exact characters in the response. Enoki-Encoder is the practical sweet spot: one encoder forward pass per fact through a 395M-parameter NLI model, sub-200ms per sentence, and you get structured triples you can log for debugging rather than opaque token scores. Enoki-Rule is worth considering if you need deterministic, non-LLM inference (audit, on-prem, no GPU). Enoki-LLM is what you’d pick for offline evaluation runs where quality dominates cost.
Code and datasets are at the anonymous repo. The EnokiQA dataset is directly usable for training or benchmarking span-level detectors, and it’s built from Wikipedia across popularity tiers with seven generator models, so it’s less prone to single-generator overfitting than earlier resources.
Anchor the intermediate representation to source characters and you get localization for free, no alignment stage required. The corollary matters more than the framework itself: once the representation carries the character spans, you can downgrade the extractor from a frontier LLM to 35 dependency-parse rules and still recover most of the quality, because verification, not decomposition, is doing the semantic work.
•
Verification only sees what extraction produced. If the extractor misses a proposition or fuses two distinct claims, no verifier calibration recovers it. This is the standard cost of explicit decomposition.
•
Incremental delta-projection assumes refinements are approximately nested. When multiple non-adjacent parts of a fact are wrong, or errors interact across facts, the flagged span will be coarser than a careful human annotation.
•
The pipeline decomposes sentence by sentence. Coreference, ellipsis, and discourse-level attribution remain only partially handled; the optional FastCoref step helps on some benchmarks and hurts on others, so there’s no free lunch for cross-sentence hallucinations.