Jina AI Releases jina-ocr-v1: A 3.4B MoE Document Parser With Built-In Speculative Decoding for Low-Budget GPUs

Jina AI, part of Elastic, has released jina-ocr-v1, an end-to-end visual document parser that turns PDFs, scans, tables, charts and invoices into Markdown in one pass. The pitch is not just accuracy. It’s speed on modest hardware, with a design meant to run on GPUs like NVIDIA’s L4 without turning every page into a small event.

The model has 3.4B total parameters, but only about 570M decoder parameters are active per token. That matters because Jina AI has baked speculative decoding into the checkpoint itself. The FastMTP head drafts text recursively for three steps, then the decoder checks those drafts greedily and keeps only what matches its own choice. The result is lossless speedup: the final text stays identical to plain greedy decoding.

Under the hood, jina-ocr-v1 is post-trained from DeepSeek-OCR and keeps its two efficiency pieces. DeepEncoder uses about 380M parameters and combines SAM, a 16x convolutional compressor and CLIP-L. A 1024×1024 page view gets squeezed from 4,096 patches down to 256 visual tokens, and a dynamic-resolution mode can add up to 9 local tiles at 100 tokens each, capping a page at 1,156 visual tokens. The decoder is a DeepSeek-3B-MoE stack with 12 layers, 64 routed experts and 2 shared experts, and output comes back as Markdown, with tables in HTML and formulas in LaTeX.

On the benchmarks Jina AI published, the model scores 91.14 on OmniDocBench v1.6 and 83.4 on olmOCR-Bench. It doesn’t lead on raw accuracy. Others score higher on each benchmark. But throughput is where it makes its case: on one A100 40 GB at concurrency 32, it parses 2.57 pages per second, the fastest of the 14 systems Jina AI measured. It also emits 1,085 output tokens per page, which Jina AI says is the shortest output among systems above 83 on their comparison.

The model is available for research and non-commercial use under CC BY-NC 4.0, with commercial use requiring contact with Jina AI. Open weights are about 6.8 GB in BF16 and run on Transformers or vLLM. Jina Reader can call it through a header, and Jina AI also offers an OpenAI-compatible endpoint plus a hosted demo for quick tests.