Package com.tnsai.llm

Interface LLMClient

All Known Implementing Classes:
FallbackLLMClient
public interface LLMClient
Base interface for LLM (Large Language Model) client implementations.

LLMClient provides a unified abstraction for interacting with different LLM providers. Agents use this interface for all chat operations, allowing easy switching between providers.

Available Implementations

ProviderClassModels
OllamaOllamaClientllama3.2, qwen3, mistral, etc.
OpenAIOpenAIClientgpt-4o, gpt-4-turbo, o1-preview
AnthropicAnthropicClientclaude-sonnet-4, claude-3-opus
GoogleGeminiClientgemini-2.5-flash, gemini-1.5-pro
AzureAzureOpenAIClientDeployment-based
GroqGroqClientllama-3.3-70b, mixtral-8x7b
OpenRouterOpenRouterClient300+ models
MistralMistralClientmistral-large, codestral
AWSBedrockClientClaude 3 via Bedrock
CohereCohereClientcommand-r, command-r+
HuggingFaceHuggingFaceClientInference API models

Quick Start


 // Using Ollama (local)
 LLMClient client = new OllamaClient("llama3.2");
 ChatResponse response = client.chat("Hello, world!");
 System.out.println(response.getContent());

 // Using OpenAI
 LLMClient client = new OpenAIClient("gpt-4o", System.getenv("OPENAI_API_KEY"));
 ChatResponse response = client.chat("What is AI?", "You are a helpful assistant");

 // With streaming
 client.streamChat("Tell me a story", Optional.empty(), Optional.empty(), Optional.empty())
     .forEach(token -> System.out.print(token));

Tool Calling (Function Calling)

LLMs can call tools/functions defined by the agent: 


 // Define tools as JSON Schema
 List<Map<String, Object>> tools = List.of(
     Map.of(
         "type", "function",
         "function", Map.of(
             "name", "get_weather",
             "description", "Get weather for a city",
             "parameters", Map.of(
                 "type", "object",
                 "properties", Map.of("city", Map.of("type", "string")),
                 "required", List.of("city")
             )
         )
     )
 );

 // Chat with tools
 ChatResponse response = client.chat(
     "What's the weather in Tokyo?",
     Optional.of("You are a weather assistant"),
     Optional.empty(),
     Optional.of(tools)
 );

 // Check for tool calls
 if (response.hasToolCalls()) {
     for (ChatResponse.ToolCall call : response.getToolCalls()) {
         System.out.println("Tool: " + call.getName());
         System.out.println("Args: " + call.getArguments());
     }
 }

Multimodal Support

Some providers (Gemini, GPT-4o) support images: 


 List<ContentPart> parts = List.of(
     new TextPart("What is in this image?"),
     ImagePart.fromFile(new File("photo.jpg"))
 );
 ChatResponse response = client.chat(parts, systemPrompt, history, tools);

Conversation History

Pass history for multi-turn conversations: 


 List<Map<String, Object>> history = List.of(
     Map.of("role", "user", "content", "My name is Alice"),
     Map.of("role", "assistant", "content", "Hello Alice! Nice to meet you.")
 );
 ChatResponse response = client.chat(
     "What's my name?",
     Optional.empty(),
     Optional.of(history),
     Optional.empty()
 );
 // Response: "Your name is Alice"

Provider Selection Guide

  • Local/Privacy: OllamaClient (runs locally)
  • Best Quality: OpenAIClient (GPT-4o) or AnthropicClient (Claude Sonnet 4)
  • Fast/Cheap: GroqClient (llama-3.3-70b)
  • Enterprise: AzureOpenAIClient (compliance, SLA)
  • Multimodal: GeminiClient or OpenAIClient
  • Code: MistralClient (Codestral) or AnthropicClient
  • RAG: CohereClient (Command R+ with connectors)

Error Handling

All implementations handle common errors:

  • Rate limiting (HTTP 429) with exponential backoff
  • Network errors with automatic retry
  • Invalid responses with descriptive exceptions
Author:
TnsAI Team
See Also:
  • ChatResponse
  • invalid reference 
    com.tnsai.llm.providers.OllamaClient
  • invalid reference 
    com.tnsai.llm.providers.OpenAIClient
  • invalid reference 
    com.tnsai.llm.providers.AnthropicClient
  • invalid reference 
    com.tnsai.llm.providers.GeminiClient

  • Method Details

    • getModel

      String getModel()
      Gets the model name being used.

    • getTemperature

      float getTemperature()
      Gets the temperature setting.

    • getTopP

      default Optional<Float> getTopP()
      Gets the top-p (nucleus sampling) setting.

    • getMaxTokens

      Optional<Integer> getMaxTokens()
      Gets the maximum tokens setting.

    • chat

      ChatResponse chat(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
      Sends a chat message and returns the response.
      Parameters:
      message - The user message
      systemPrompt - Optional system prompt
      history - Optional conversation history
      tools - Optional list of available tools for function calling
      Returns:
      The LLM response as ChatResponse (contains content and optional tool calls)

    • chat

      default ChatResponse chat(List<ContentPart> messageParts, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
      Sends a multimodal chat message (text + images/files).
      Parameters:
      messageParts - List of content parts (TextPart, ImagePart)
      systemPrompt - Optional system prompt
      history - Optional conversation history
      tools - Optional list of available tools
      Returns:
      The LLM response

    • chat

      default ChatResponse chat(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools, float temperature)
      Sends a chat message with temperature override.
      Parameters:
      message - The user message
      systemPrompt - Optional system prompt
      history - Optional conversation history
      tools - Optional list of available tools for function calling
      temperature - Temperature override for this request (0.0-2.0)
      Returns:
      The LLM response as ChatResponse

    • streamChat

      Stream<String> streamChat(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
      Sends a chat message and streams the response token by token.

      Important: The returned Stream holds an HTTP connection open. Callers must close the stream when done, ideally via try-with-resources: 

      
 try (Stream<String> tokens = client.streamChat(msg, sys, hist, tools)) {
     tokens.forEach(System.out::print);
 }
      Failing to close the stream will leak HTTP connections.
      Parameters:
      message - The user message
      systemPrompt - Optional system prompt
      history - Optional conversation history
      tools - Optional list of available tools for function calling
      Returns:
      Stream of response tokens — caller must close

    • streamChat

      default Stream<String> streamChat(List<ContentPart> messageParts, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
      Sends a multimodal chat message and streams the response.

    • chat

      default ChatResponse chat(String message)
      Simplified chat without system prompt or history.

    • chat

      default ChatResponse chat(String message, String systemPrompt)
      Simplified chat with system prompt.

    • chat

      default ChatResponse chat(ChatRequest request)
      Sends a chat request using the unified ChatRequest model.

      Dispatches to the appropriate chat method based on request properties:

      Parameters:
      request - the chat request
      Returns:
      the LLM response

    • streamChat

      default Stream<String> streamChat(ChatRequest request)
      Streams a chat response using the unified ChatRequest model.
      Parameters:
      request - the chat request
      Returns:
      stream of response tokens

    • streamChatWithSpec

      default Stream<ChatChunk> streamChatWithSpec(ChatRequest request)
      Streams a chat response with rich metadata using the unified ChatRequest model.
      Parameters:
      request - the chat request
      Returns:
      stream of ChatChunk objects

    • streamChatWithHandler

      default void streamChatWithHandler(ChatRequest request, Consumer<ChatChunk> handler)
      Streams a chat response via callbacks using the unified ChatRequest model. Supports both text streaming and tool calls.
      Parameters:
      request - the chat request
      handler - callback that receives ChatChunk events

    • streamChatWithHandler

      default void streamChatWithHandler(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools, Consumer<ChatChunk> handler)
      Streams a chat response via callbacks, supporting both text deltas and tool calls.

      This method combines streaming and tool calling in a single operation. Providers can override this to implement true SSE-based streaming with tool call detection. The default implementation falls back to chat(String, Optional, Optional, Optional) and emits the result as chunks.

      Emitted chunk types:

      Parameters:
      message - The user message
      systemPrompt - Optional system prompt
      history - Optional conversation history
      tools - Optional list of available tools
      handler - Callback that receives ChatChunk events

    • streamChatWithSpec

      default Stream<ChatChunk> streamChatWithSpec(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
      Sends a chat message and streams the response with rich metadata.

      Unlike streamChat(java.lang.String, java.util.Optional<java.lang.String>, java.util.Optional<java.util.List<java.util.Map<java.lang.String, java.lang.Object>>>, java.util.Optional<java.util.List<java.util.Map<java.lang.String, java.lang.Object>>>), this method returns ChatChunk objects that include metadata like token count, chunk index, finish reason, etc.

      Example Usage

      
 client.streamChatWithSpec("Tell me a story", Optional.empty(), Optional.empty(), Optional.empty())
     .forEach(chunk -> {
         if (chunk.isContent()) {
             System.out.print(chunk.getContent());
         }
         if (chunk.isDone()) {
             System.out.println("\nTotal tokens: " + chunk.getTokenCount().orElse(0));
         }
     });
      Parameters:
      message - The user message
      systemPrompt - Optional system prompt
      history - Optional conversation history
      tools - Optional list of available tools for function calling
      Returns:
      Stream of ChatChunk objects with content and metadata

    • listModels

      default List<ModelInfo> listModels()
      Lists available models from the provider.

      This method queries the provider's API to get a list of available models. For cloud providers like OpenAI, this calls the models API endpoint. For local providers like Ollama, this lists installed models.

      Example Usage

      
 LLMClient client = new OpenAIClient("gpt-4o", apiKey);
 List<ModelInfo> models = client.listModels();
 for (ModelInfo model : models) {
     System.out.printf("%s (owned by %s)%n", model.id(), model.ownedBy());
 }
      Returns:
      List of available models from the provider
      Throws:
      RuntimeException - if the API call fails

    • getProviderName

      default String getProviderName()
      Gets the provider name for this client.
      Returns:
      The provider name (e.g., "openai", "anthropic", "ollama")

    • getCapabilities

      default LLMCapabilities getCapabilities()
      Returns the capabilities of this LLM client/model.

      Capabilities include information about:

      • Supported features (vision, function calling, structured output)
      • Token limits (context window, max output)
      • Supported modalities (text, image, audio)
      • Cost information (input/output cost per 1K tokens)

      Example Usage

      
 LLMCapabilities caps = client.getCapabilities();

 if (caps.supportsVision()) {
     // Use multimodal features
 }

 if (caps.getMaxInputTokens() < estimatedTokens) {
     // Truncate input
 }
      Returns:
      The capabilities of this LLM