AL Alfred 2026-09-04 4 min read

Jina-OCR-v1: 2.57 Pages/s, 2x Faster OCR Parsing on L4

Document parsing is the quiet bottleneck behind every RAG pipeline and PDF ingestion job — and most state-of-the-art OCR models are too expensive to run where the documents actually live. Jina-OCR-v1 attacks exactly that: an end-to-end parsing model built to serve on low-budget GPUs, pairing DeepSeek-OCR's compressed vision encoder and 3B mixture-of-experts decoder with a speculative decoding head that doubles decode speed on an NVIDIA L4 — losslessly. It posts 91.14 on OmniDocBench v1.6, 83.4 on olmOCR-Bench, and the highest page throughput in its comparison at 2.57 pages per second. Weights are public per the paper.

The Problem: OCR Accuracy Stopped Being the Bottleneck

Modern vision-language OCR models read documents well — GPT-4o-class vision and DeepSeek-OCR pushed past the accuracy ceiling everyone complained about three years ago. What didn't improve is the cost profile: full-parameter VLMs doing token-by-token autoregressive decoding are slow and memory-hungry, which hurts most at small batch sizes and on the cheap GPUs where document workloads actually run. If you're a startup ingesting 100k PDFs a day, throughput per dollar decides whether OCR is a feature or a line item. Jina-OCR-v1 is a serving-efficiency paper disguised as a model release.

The Method: A 3B MoE Decoder with a Lossless Speculative Head

The architecture reuses DeepSeek-OCR's compressed vision encoder (which shrinks visual tokens before the language model sees them) and its 3B Mixture-of-Experts decoder, activating only about 570M parameters per token — the trick that keeps inference cheap without collapsing quality. On top of that, Jina adds a FastMTP speculative decoding head: a single draft block shared recursively across K=3 prediction steps, so one small module drafts several tokens ahead. Greedy verification against the real model makes the speedup lossless — the accepted output distribution is identical to greedy decoding, just produced faster. Post-training is where this gets interesting: instruction alignment and robustness fine-tuning on hard documents, then GRPO reinforced with dense verifiable rewards — deterministic formula, table, and structural checks that award partial credit instead of a binary pass/fail. Training data mixes cleaned public corpora with targeted synthetic pages.

flowchart LR
    A[Document Page] --> B[Compressed Vision Encoder]
    B --> C[3B MoE Decoder
~570M active params/token]
    C --> D[FastMTP Draft Head
shared block x K=3]
    D --> E[Greedy Verification]
    E -->|accept| F[Markdown + Structure Out]
    E -->|reject| C

The Results: 91.14 OmniDocBench, 2.57 Pages/s, 2x L4 Decode

At the default dynamic-resolution setting, Jina-OCR-v1 scores 91.14 on OmniDocBench v1.6 and 83.4 on olmOCR-Bench — squarely in the class of document parsers you'd actually ship. The throughput headline: 2.57 pages per second, the highest of any model in the paper's comparison. On an NVIDIA L4 — the archetypal low-budget inference GPU — FastMTP doubles decoding speed over greedy autoregressive decoding. The combination is the point: accuracy that doesn't embarrass you, at roughly half the wall-clock cost of baseline decoding on hardware that rents for a few dollars an hour.

Limitations: What the Numbers Don't Say

This is a 2x decode-speedup claim, not a 2x end-to-end pipeline claim — page throughput already includes parsing, but your own preprocessing, batching, and serving overhead still apply. "Lossless" refers to the greedy decoding distribution: the Dense Verifiable Rewards post-training only generalizes as far as its deterministic checks do — a formula-heavy or oddly structured page is exactly where check coverage is thinnest. The accuracy figures cover two benchmarks at one dynamic-resolution setting; hard, noisy, or rotated scans are the robustness fine-tuning target, not a guarantee. And "low-budget GPU" is relative: a 3B MoE is light on FLOPs, but it still needs enough memory to hold the decoder plus the draft block, which rules out most CPU-only or edge deployments. Finally, the architecture inherits DeepSeek-OCR's design — the differentiation is the head, the reward shaping, and the serving target, not a new decoding paradigm.

Why Builders Should Care

If you're serving extraction over PDFs, invoices, or academic papers, this is the first open-weight model in a while that optimizes for your constraint — cost per page on modest hardware — instead of chasing a leaderboard. The testable claims are concrete: stick it on an L4, measure pages per second yourself, and check whether 91 on OmniDocBench clears your quality bar. The recipe matters beyond the model, too. Lossless speculative decoding plus GRPO on dense verifiable rewards is a template other domains can copy: any task where output can be checked deterministically (tables, forms, code blocks, math) can be trained for speed without sacrificing correctness. That's the part worth stealing even if you never touch OCR.

Related Posts