Package com.tnsai.llm.providers

Class NvidiaNIMClient

java.lang.Object

com.tnsai.llm.providers.AbstractLLMClient

com.tnsai.llm.providers.AbstractOpenAICompatibleClient

com.tnsai.llm.providers.NvidiaNIMClient

All Implemented Interfaces:

com.tnsai.llm.LLMClient

public class NvidiaNIMClient
extends AbstractOpenAICompatibleClient

NVIDIA NIM (NVIDIA Inference Microservices) LLM client.

NIM exposes the same OpenAI-compatible chat completions API on two deployment shapes: the hosted catalog at https://integrate.api.nvidia.com/v1 (cloud) and any locally-running NIM container (self-host). This client targets both via a baseUrl parameter — pass the container's URL to talk to a local NIM, leave it null to fall through to the cloud catalog endpoint.

Supported models (cloud catalog at time of writing):

• meta/llama-3.1-405b-instruct — Llama 3.1 405B (flagship)
• meta/llama-3.1-70b-instruct — Llama 3.1 70B
• meta/llama-3.1-8b-instruct — Llama 3.1 8B (fast / cheap)
• mistralai/mixtral-8x7b-instruct-v0.1 — Mixtral 8x7B
• mistralai/mistral-7b-instruct-v0.3 — Mistral 7B
• nvidia/nemotron-4-340b-instruct — Nemotron 4 340B (NVIDIA-trained)
• nvidia/llama-3.1-nemotron-70b-instruct — Llama+Nemotron tune

The catalog evolves quickly; check the live list at build.nvidia.com. Self-hosted containers ship one model per container — pass whatever model name the container advertises.

Configuration via environment variables:

• NVIDIA_API_KEY — Required for the cloud catalog; self-hosted NIMs typically accept any value (or none) depending on how they're deployed.
• NVIDIA_BASE_URL — Optional override, defaults to https://integrate.api.nvidia.com/v1. Point at a local NIM container (e.g. http://localhost:8000/v1) for self-hosted use.

See Also:

• NIM API Documentation

Constructor Summary

Constructors

Constructor	Description
NvidiaNIMClient(String model)
NvidiaNIMClient(String model, float temperature)
NvidiaNIMClient(String model, float temperature, Float topP, Integer maxTokens)
NvidiaNIMClient(String model, float temperature, Integer maxTokens)
NvidiaNIMClient(String baseUrl, String model, float temperature, Float topP, Integer maxTokens, String apiKey)
NvidiaNIMClient(String baseUrl, String model, float temperature, Integer maxTokens, String apiKey)

Method Summary

Methods inherited from class com.tnsai.llm.providers.AbstractOpenAICompatibleClient

chat, streamChat

Methods inherited from class com.tnsai.llm.providers.AbstractLLMClient

getBaseUrl, getCallTimeoutMs, getMaxTokens, getModel, getTemperature, getTopP

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.tnsai.llm.LLMClient

chat, chat, chat, chat, chat, getCapabilities, getProviderName, listModels, streamChat, streamChat, streamChatWithHandler, streamChatWithHandler, streamChatWithSpec, streamChatWithSpec

Constructor Details

NvidiaNIMClient

public NvidiaNIMClient(String model)

NvidiaNIMClient

public NvidiaNIMClient(String model,
 float temperature)

NvidiaNIMClient

public NvidiaNIMClient(String model,
 float temperature,
 Integer maxTokens)

NvidiaNIMClient

public NvidiaNIMClient(String model,
 float temperature,
 Float topP,
 Integer maxTokens)

NvidiaNIMClient

public NvidiaNIMClient(String baseUrl,
 String model,
 float temperature,
 Integer maxTokens,
 String apiKey)

NvidiaNIMClient

public NvidiaNIMClient(String baseUrl,
 String model,
 float temperature,
 Float topP,
 Integer maxTokens,
 String apiKey)