Package com.tnsai.llm.providers

Class DeepInfraClient

java.lang.Object

com.tnsai.llm.providers.AbstractLLMClient

com.tnsai.llm.providers.AbstractOpenAICompatibleClient

com.tnsai.llm.providers.DeepInfraClient

All Implemented Interfaces:

com.tnsai.llm.LLMClient

public class DeepInfraClient
extends AbstractOpenAICompatibleClient

DeepInfra LLM client.

DeepInfra hosts open models on commodity GPUs and competes on price — typically the cheapest hosted Llama-70B option. The API is OpenAI-compatible at https://api.deepinfra.com/v1/openai (note the /v1/openai suffix — DeepInfra's native API lives at /v1/inference; we target the OpenAI-compatible shim so the wire format matches every other provider in this module).

Supported models (at time of writing — DeepInfra's serverless catalogue evolves quickly, check the live list at deepinfra.com/models):

• meta-llama/Llama-3.3-70B-Instruct — Llama 3.3 70B (cost leader)
• meta-llama/Meta-Llama-3.1-405B-Instruct — Llama 3.1 405B
• meta-llama/Meta-Llama-3.1-8B-Instruct — Llama 3.1 8B
• mistralai/Mixtral-8x7B-Instruct-v0.1 — Mixtral 8x7B
• Qwen/Qwen2.5-72B-Instruct — Qwen 2.5 72B
• deepseek-ai/DeepSeek-V3 — DeepSeek V3

Configuration via environment variables:

• DEEPINFRA_API_KEY — Required.
• DEEPINFRA_BASE_URL — Optional override, defaults to https://api.deepinfra.com/v1/openai.

See Also:

• DeepInfra OpenAI-Compatible API

Constructor Summary

Constructors

Constructor	Description
DeepInfraClient(String model)
DeepInfraClient(String model, float temperature)
DeepInfraClient(String model, float temperature, Float topP, Integer maxTokens)
DeepInfraClient(String model, float temperature, Integer maxTokens)
DeepInfraClient(String baseUrl, String model, float temperature, Float topP, Integer maxTokens, String apiKey)
DeepInfraClient(String baseUrl, String model, float temperature, Integer maxTokens, String apiKey)

Method Summary

Methods inherited from class com.tnsai.llm.providers.AbstractOpenAICompatibleClient

chat, streamChat

Methods inherited from class com.tnsai.llm.providers.AbstractLLMClient

getBaseUrl, getCallTimeoutMs, getMaxTokens, getModel, getTemperature, getTopP

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.tnsai.llm.LLMClient

chat, chat, chat, chat, chat, getCapabilities, getProviderName, listModels, streamChat, streamChat, streamChatWithHandler, streamChatWithHandler, streamChatWithSpec, streamChatWithSpec

Constructor Details

DeepInfraClient

public DeepInfraClient(String model)

DeepInfraClient

public DeepInfraClient(String model,
 float temperature)

DeepInfraClient

public DeepInfraClient(String model,
 float temperature,
 Integer maxTokens)

DeepInfraClient

public DeepInfraClient(String model,
 float temperature,
 Float topP,
 Integer maxTokens)

DeepInfraClient

public DeepInfraClient(String baseUrl,
 String model,
 float temperature,
 Integer maxTokens,
 String apiKey)

DeepInfraClient

public DeepInfraClient(String baseUrl,
 String model,
 float temperature,
 Float topP,
 Integer maxTokens,
 String apiKey)