Nemotron-3.5-Rerank-8B-BF16

What NVIDIA Nemotron-3.5-Rerank-8B-BF16 is

NVIDIA Nemotron-3.5-Rerank-8B-BF16 is a specialized multimodal reranker rather than a general-purpose conversational language model. Its job is to compare a query with a candidate document and estimate how relevant that document is. The document may be plain text, an RGB image, or a combination of text and image content.

The model is an 8.38-billion-parameter cross-encoder. In a cross-encoder, the query and candidate document are evaluated together, allowing the model to examine their relationship directly. A retrieval system can first find a larger group of possible matches using a fast search method, then pass those candidates to Nemotron-3.5-Rerank-8B-BF16 for more careful second-stage ordering.

The model returns a one-dimensional floating-point raw logit for each query-document pair. Higher scores indicate greater relevance. These are ranking scores, not generated answers, explanations, embeddings, or confidence percentages.

Where it fits in NVIDIA's lineup

NVIDIA positions this model within its Nemotron Retriever family, alongside retrieval-focused models for embedding, reranking, and document processing. The current model is specifically the reranking component: it improves the order of results produced by an earlier retrieval stage.

The canonical open-weight checkpoint is named nvidia/Nemotron-3.5-Rerank-8B-BF16 and is available through Hugging Face. NVIDIA also provides access through a NIM endpoint using the model identifier nvidia/nemotron-3.5-rerank-8b. NVIDIA Build lists a free development endpoint, subject to rate limits. Production inference pricing is not publicly specified in the supplied documentation.

How it works in a retrieval pipeline

A typical use case has two stages. First, a search engine, vector database, or embedding-based retriever gathers a manageable set of candidate documents. Second, Nemotron-3.5-Rerank-8B-BF16 examines each candidate alongside the user query and assigns a relevance score. The application then sorts the candidates by score and sends the best results to a downstream language model or displays them directly.

For example, an enterprise search system could retrieve several pages from a technical manual for the query “maximum operating temperature.” The reranker can compare the query with the text and visual content of each page, helping prioritize the page containing the relevant specification table rather than simply choosing a page with frequent keyword matches.

The same pattern applies to retrieval-augmented generation (RAG). Reranking can reduce the amount of irrelevant context sent to a response-generating model. This may improve the usefulness of retrieved evidence, but Nemotron-3.5-Rerank-8B-BF16 itself does not write the final answer.

Supported inputs and outputs

This makes the model multimodal on input, but not multimodal in output. It does not produce images, audio, or video. Its direct result is a numerical relevance score.

Context and input limits

The model documentation states a maximum context length of 32,768 tokens. However, the hosted NVIDIA reranking API documents a more specific operational limit: each query-document pair, including image tokens, must fit within 10,240 tokens. Applications using the hosted endpoint should therefore treat 10,240 tokens per pair as the practical service constraint unless NVIDIA documentation for the selected deployment states otherwise.

This limit applies to the complete pair rather than only the text query. Long documents, OCR text, captions, and image-derived tokens can all consume the available budget. In practice, systems may need to split long documents into passages or select relevant pages before reranking. The supplied research does not specify a maximum output-token setting because the model produces scores instead of generated token sequences.

Main strengths

• Multimodal document matching: the model can evaluate visual documents such as charts, tables, infographics, datasheets, and schematics in addition to ordinary text.
• Direct query-document comparison: as a cross-encoder, it evaluates the relationship between the query and candidate rather than producing a standalone representation only.
• Multilingual retrieval: NVIDIA reports evaluation for text retrieval across 34 languages and image retrieval across 18 languages.
• Useful second-stage filtering: it can refine a broad initial search before evidence is passed to a generative model or shown to a user.
• Open-weight availability: the BF16 checkpoint can be used as an open-weight model under the OpenMDW License Agreement 1.1, subject to that license's terms.
• Fine-tuning support: NVIDIA documentation and the model card describe fine-tuning support for multimodal reranking.

Its strongest distinction is the combination of reranking and visual-document understanding. A text-only reranker may be sufficient when all important information has already been extracted into text, but this model is intended for cases where the layout or visual content remains important.

Limitations and trade-offs

Nemotron-3.5-Rerank-8B-BF16 is not a substitute for a chat model, question-answering model, embedding model, or document parser. It does not generate natural-language responses, create embeddings as its documented output, perform web search, or call tools and functions. A complete application must provide the initial retrieval stage and, when needed, a separate model for answer generation.

Its 8.38-billion-parameter size and cross-encoder design can make it more computationally expensive than lightweight retrieval methods when many candidates must be scored. Reranking hundreds or thousands of documents individually may increase latency and infrastructure cost. The usual trade-off is to retrieve a smaller candidate set quickly, then apply the reranker only to the most promising results.

Image inputs also require careful handling. The hosted API's 10,240-token query-document-pair limit includes image tokens, so a high-resolution or information-dense document may need preprocessing, page selection, or chunking. The supplied research does not provide a benchmark-based speed score, cost score, or general reasoning score, so no comparative performance claim should be inferred from the parameter count alone.

Reasoning, coding, and agent workflows

This model should not be evaluated as a general reasoning or coding assistant. It does not solve problems in the conversational sense or write software. Its useful form of “reasoning” is relevance discrimination: determining which candidate best matches a query, including when relevant evidence appears in a visual document.

It can still support agentic systems indirectly. An agent can use a retrieval pipeline containing this reranker to choose which manuals, reports, charts, or schematics to inspect before taking a later action. The model itself has no documented tool-use or function-calling capability, so orchestration, search, document loading, and actions must be implemented by the surrounding application.

Pricing and availability

NVIDIA Build lists a free development endpoint, subject to rate limits. The supplied research does not specify a production price for hosted inference, and the open-weight checkpoint does not have a stated per-token price in the reviewed materials. Output pricing is not applicable in the usual generative-model sense because the model returns relevance logits rather than generated tokens.

Organizations deploying the checkpoint themselves should account for infrastructure, storage, image-processing, and operational costs. The research does not provide a standardized speed or hardware-cost comparison, so deployment economics depend on the selected serving environment and candidate volume.

When to choose Nemotron-3.5-Rerank-8B-BF16

Choose this model when the retrieval stage needs to rank candidate documents using both language and visual information. It is particularly suitable for:

• multimodal RAG over scanned reports, charts, tables, and technical diagrams;
• enterprise search where document layout carries meaning;
• multilingual text retrieval and visual-document search;
• second-stage ranking after keyword, vector, or hybrid retrieval;
• agentic retrieval pipelines that must select evidence before another model generates an answer;
• custom retrieval systems where fine-tuning a reranker is useful.

A smaller or text-only reranker may be more appropriate when the corpus contains only clean text, latency is the overriding requirement, or the system must score a very large candidate set at low cost. An embedding model is a better fit for creating vectors and performing initial nearest-neighbor retrieval. A generative language model is more appropriate when the required output is an explanation, summary, answer, or code. A dedicated document-parsing system may be preferable when the primary requirement is structured extraction rather than relevance ranking.

Bottom line

NVIDIA Nemotron-3.5-Rerank-8B-BF16 is best understood as a specialized relevance judge for multimodal retrieval. Its value comes from scoring text and visual documents against a query, with support for multilingual retrieval and use in RAG or agentic search pipelines. The main trade-off is that it adds a relatively substantial second-stage model to the retrieval stack and returns scores rather than answers. For systems that need accurate ranking of text-and-image evidence, that specialization is the reason to consider it; for generation, embeddings, or simple low-latency text search, another model type is likely a better fit.