OvisOCR2 is a 0.8B end-to-end document-to-Markdown model that beats larger pipeline parsers on OmniDocBench v1.6 by pairing a source-of-truth synthetic data engine with a 4B Group Relative Policy Optimization (GRPO) teacher distilled into the 0.8B student via On-Policy Distillation.
If you ship a document-ingestion service (invoices, scientific PDFs, scanned reports flowing into a RAG index), you probably run a pipeline today: a layout detector crops regions, then specialized recognizers handle text, tables, formulas, and figures, then a merger reassembles a Markdown page. Two models, two GPUs, error accumulation. The pipeline misses a table row at layout time and no downstream recognizer can recover it. The leading systems on public leaderboards. PaddleOCR-VL-1.6, MinerU2.5-Pro, GLM-OCR. all work this way. End-to-end vision-language parsers (one model, image in, Markdown out) are simpler to deploy but have historically trailed on quality. OvisOCR2 is the paper arguing that gap has closed, and in fact reversed, at surprisingly small scale.
The backbone is Qwen3.5-0.8B post-trained for page-to-Markdown. The interesting parts are the data pipeline and the training ladder.
For data, they run two tracks. The real-document track takes page images, runs them through existing strong parsers (PaddleOCR-VL-1.5 and MinerU2.5-Pro), and converts their structured JSON outputs into a normalized Markdown schema through hand-written rules (heading levels from numbering patterns, tables as HTML with border=1, formulas as $...$ and $$...$$, figure regions as <img> tags with bounding boxes rescaled to [0,1000)). Subsets are then spot-checked by humans and dropped if error rates are high. The synthetic track inverts the usual synthesis flow: instead of rendering a page and then labeling it, they start from an HTML template (seeded from mined hard cases like merged-cell tables or multi-column layouts), diversify it with an agent, render the image with Playwright, and derive the Markdown label directly from the same HTML DOM. The label is deterministic because it never depends on parsing the rendered pixels.
Training runs on two branches. Both a 0.8B and a 4B model get supervised fine-tuning on the full mixture. Then the 4B branch gets reinforcement learning with Group Relative Policy Optimization (GRPO): sample several Markdown outputs per page, score each with a composite reward (text edit distance, Character Detection Matching (CDM) for formulas, Tree-Edit-Distance Similarity (TEDS) for tables), average only over components present in the reference, and update toward the better samples in each group. The authors note that running RL directly on the 0.8B model was unstable (KL divergence blew up, table quality regressed), so instead the RL-tuned 4B becomes a teacher. The 0.8B student then goes through On-Policy Distillation: the student generates a full page, and at each token position the teacher scores the student’s top-k candidates. The loss is a reverse-KL restricted to that top-k support, which keeps memory at O(T·k) instead of O(T·V) for long pages. Finally, several 0.8B variants trained with different data mixes are averaged into one set of weights (Model soups-style fusion).
# 4B teacher, RL with GRPO
for page in hard_pages:
outs = [policy_4b.sample(page) for _ in range(G)]
rewards = [avg_over_available(text_ed, cdm, teds) for o in outs]
policy_4b.grpo_update(outs, rewards) # group-relative, no value net
# 0.8B student, on-policy distillation from 4B teacher
for page in pages:
y = policy_0_8b.sample(page) # student rollout
for t in positions(y):
S = topk(policy_0_8b.probs(y[:t])) # student top-k support
q = teacher_4b.probs(y[:t], S) # teacher scores those tokens
loss += KL(renorm(policy_0_8b, S) || renorm(q, S))
The prevailing assumption in document AI is that page parsing is too heterogeneous for one model. layout, tables, formulas, and reading order each need a specialist, wired together by a pipeline. This paper argues the opposite. A single 0.8B end-to-end model, if you feed it deterministically-labeled synthetic pages and shape its behavior through structural rewards, can outperform pipelines built from larger specialized models. The load-bearing evidence is not the headline leaderboard number but the missing-rate comparison on complex tables: pipelines lose 13\u201317% of tables at layout time, while OvisOCR2 loses ~8%, an error class that end-to-end generation sidesteps entirely.
•
On the complex-table subset of their in-house benchmark, pipeline methods have missing rates of 13\u201317% (tables silently dropped during layout parsing). OvisOCR2’s missing rate is 7.96%. This is the number that most directly proves the thesis: pipeline-stage errors are structurally irrecoverable, and end-to-end generation avoids them.
•
On OmniDocBench v1.6, overall score 96.58, first place on a leaderboard the paper notes was previously led by pipeline methods. Improvement over the previous best end-to-end method is +1.84 points. Best-in-table on text edit distance, formula Character Detection Matching (CDM), Tree-Edit-Distance Similarity (TEDS)-S, and reading-order edit distance.
•
On PureDocBench, Avg3 of 75.06, first overall. But on the “Real” track (phone photos, photocopies, screen recaptures) OvisOCR2 trails general VLMs like Gemini-3.1-Pro and Qwen3.5-122B-A10B. The authors flag degraded-image robustness as open work.
•
Direct 0.8B RL was unstable: the training-curve figure shows KL divergence rising and validation table Tree-Edit-Distance Similarity (TEDS) falling in late training, versus the 4B branch which stays stable. This is why On-Policy Distillation exists in the recipe rather than just running GRPO on the small model.
•
The in-house benchmark advantage holds across easy/medium/hard tiers, and on the handwriting subset OvisOCR2 leads overall despite one competitor scoring higher on table Tree-Edit-Distance Similarity (TEDS) alone.
Reach for OvisOCR2 when you’re building document ingestion for RAG or search over PDFs and your current stack is a layout detector plus separate table and formula recognizers. A single 0.8B model replaces that stack, outputs Markdown in one pass, and the failure mode of “the layout stage dropped a table and nothing downstream noticed” mostly goes away. The Markdown target format (HTML tables inline, LaTeX-delimited formulas, <img> tags with normalized bounding boxes) is directly consumable by chunkers.
The model weights are on Hugging Face. The paper doesn’t release the data engine, the synthetic HTML templates, or training code. The two public evaluation benchmarks (OmniDocBench v1.6 and PureDocBench) are independently available if you want to reproduce numbers. There is no released in-house benchmark.
•
Degraded real-world images are the weak spot. On PureDocBench’s Real track (phone captures, photocopies, screen photos), general large VLMs beat OvisOCR2. If your input distribution is mostly user-uploaded photos rather than born-digital PDFs, the advantage narrows or flips.
•
The recipe depends on other strong parsers. Real-document training labels come from PaddleOCR-VL-1.5 and MinerU2.5-Pro outputs, filtered by rules and human spot-checks. This is a distillation-flavored setup dressed as end-to-end training. Reproducing it from scratch without access to those parsers’ outputs is nontrivial, and the ceiling is bounded by their quality on the un-spot-checked mass of data.
•
The 4B\u21920.8B ladder is load-bearing. The authors explicitly show direct 0.8B RL is unstable. If you tried to skip the teacher and just run GRPO on a small model with the same rewards, you’d likely get the worse curve in Figure 4, not the reported numbers.
When structural rewards can be computed cheaply and labels can be made deterministic, an end-to-end small model beats a pipeline of larger specialists. The trick isn’t the architecture, it’s refusing to let parser noise into your labels (synthesize from HTML, not from rendered pixels) and refusing to run high-variance RL on the deployment-size model (train the teacher big, distill on-policy into the student).