NVIDIA Nemotron 3.5 adds visual retrieval and document parsing

NVIDIA has expanded its Nemotron retrieval collection with models for finding documents, ranking candidate evidence and extracting page content. Its October 21 overview brings these components together as a toolkit for agents working with text, tables and images. They are not interchangeable chat models. Each performs a different job before an answering model uses the evidence.

The release dates need separating from that overview. NVIDIA lists October 20 for its Embed and Rerank checkpoints, while the Parse model card records October 17. The cards do not establish an exact release time for the October 20 models. This is a combined look at the available retrieval workflow, not a claim that every component first appeared on October 21.

Finding a page and answering a question are different jobs

Nemotron 3.5 Embed 8B turns a text query and document content into vectors that can be compared. Documents can supply text, a page image or both. Queries accept text. That distinction matters for a search interface that handles illustrated reports but still expects the user to type a question.

The 8B checkpoint has 8.38 billion parameters and produces 4,096 dimensional vectors. NVIDIA also supports shorter vectors, such as 1,024 or 512 dimensions, provided they are normalized again after truncation. Its language component remains frozen from the preceding Nemotron 3 Embed 8B model, and NVIDIA says their text embeddings are identical. Existing text indexes may therefore be reusable rather than automatically needing a complete rebuild.

The smaller Embed 1B BF16 checkpoint actually contains 1.55 billion parameters. It produces 2,048 dimensional vectors and shares the text embedding space of its corresponding Nemotron 3 predecessor. Its maximum context is 32,768 tokens, including image tokens, with a default image longest edge of 1,120 pixels. These are input and representation choices to test against the pages a system really stores.

Rerank 8B operates after that search. It reads each query together with one candidate document and returns a relevance score, rather than another searchable document vector. Its 8.38 billion parameters support text, page images or combined content. Higher output logits indicate stronger relevance to the query. They should not be presented to readers as calibrated probabilities that an answer is correct.

Smaller downloads do not remove hardware constraints

The Embed 1B NVFP4 variant quantizes eligible layers in both the vision and language components. NVIDIA specifies vLLM and hardware with NVFP4 support, such as Blackwell GPUs. Developers needing Transformers or Sentence Transformers are directed to the BF16 version instead. A compact checkpoint name alone does not establish compatibility with an existing workstation.

NVIDIA says the BF16 and NVFP4 versions generally share an embedding space, but recommends checking retrieval quality on representative samples before switching. Its documented serving combination uses vLLM 0.32.0 and Transformers 5.19.0. Startup includes kernel tuning and CUDA graph capture, so initial startup time should not be confused with normal query latency.

The BF16 instructions have a separate compatibility detail. Their Transformers examples use version 5.20.0, while the vLLM recipe uses a separate environment with Transformers 5.19.0. NVIDIA warns that Flash Attention 2 with Transformers 5.17.0, 5.18.0 and 5.19.0 can cause CUDA errors or incorrect image outputs. Following the runtime specific recipe is more useful than combining commands from different examples.

Parse preserves information that plain text can lose

Nemotron 3.5 Parse is the document extraction component. Its approximately 631 million parameters produce text with semantic element classes, bounding boxes and reading order. Postprocessing can render Markdown with HTML tables and structured blocks. HTML and hierarchical JSON can preserve merged cells, while flat exports such as CSV flatten spans. Choose the output format around the downstream task, not merely the most familiar extension.

The model card also puts an important limit on its OCR evidence. MOSCAR uses simply rendered synthetic text and measures character overlap without considering character order. That result does not establish reliable extraction from handwriting, damaged scans or complex layouts. For those documents, inspect the actual extracted rows and page locations rather than treating a high OCR score as a blanket guarantee.

What the efficiency evidence does and does not show

NVIDIA reports that adding its reranker reduced agent inference tokens per query by about 14 percent with Embed 8B and 13 percent with Embed 1B, while improving retrieval accuracy. The experiments held the answering model, tools, tasks, corpus and execution policy fixed while changing retrieval. That is a useful controlled comparison, but it measures agent token consumption rather than every cost in operating the retrieval system.

For a deployment decision, measure indexing cost, retrieval latency, reranking latency and final answer quality separately. A system may send fewer tokens to its answering model while doing more work in a separate retrieval stage. ByteForward has not run these checkpoints or reproduced NVIDIA’s results.

A practical evaluation should include questions whose evidence lives in charts, tables and ordinary paragraphs, plus questions the collection cannot answer. Keep the retrieved page and its source location available for inspection. Compare the final answers as well as the search rankings, because finding a relevant page is not the same as interpreting it correctly.

The model cards identify OpenMDW version 1.1 as the governing model license and describe commercial use as available. Check the exact checkpoint and its component notices before deployment. These retrieval models are separate from the answer generating models in ByteForward’s Nemotron Super VL and Bolt coverage. The useful choice is which retrieval stages a workload needs, not whether one family name can replace the entire application.

Illustrative photograph by cottonbro studio via Pexels under the Pexels License.