com.tnsai.llm.LLMClientFactory

public final class LLMClientFactory extends Object

Factory for creating LLM clients from configuration.

This factory enables annotation-driven LLM initialization by creating the appropriate LLMClient implementation based on provider name.

Supported Providers

ollama - Local/cloud Ollama models
openai - OpenAI GPT models
anthropic - Anthropic Claude models
gemini - Google Gemini models
groq - Groq inference
mistral - Mistral AI models
cohere - Cohere models
openrouter - OpenRouter aggregator
azure - Azure OpenAI
bedrock - AWS Bedrock
huggingface - HuggingFace Inference
zhipu - ZhipuAI (GLM models)
minimax - MiniMax AI
cerebras - Cerebras WSE-3 inference (ultra-fast)
nvidia - NVIDIA NIM (cloud catalog + self-hosted containers)
deepinfra - DeepInfra (cheap open-model hosting)
fireworks - Fireworks.ai (open-model hosting + FireFunction)
together - Together.ai (open model hosting)
xai - xAI Grok models
watsonx - IBM watsonx.ai (Granite + hosted Llama/Mistral)
vertexai - Google Cloud Vertex AI (enterprise Gemini)
replicate - Replicate model marketplace (predict/poll)
deepseek - DeepSeek V3 chat + R1 reasoning
lmstudio - LM Studio local server (OpenAI-compatible)
perplexity - Perplexity search-augmented LLM
databricks - Databricks Mosaic AI Model Serving
qwen - Alibaba DashScope (Qwen family)
llamacpp - llama.cpp server (edge / on-device)
vllm - vLLM self-hosted server (OpenAI-compatible)
hunyuan - Tencent Hunyuan family
yi - 01.AI Yi family

Usage with @LLMSpec Annotation


 @RoleSpec(
     name = "ResearchAgent",
     llm = @LLMSpec(
         provider = "ollama",
         model = "llama3",
         temperature = 0.7f
     )
 )
 public class ResearchRole extends Role {
     // LLM is automatically initialized from annotation
 }

Programmatic Usage


 // Simple creation
 LLMClient client = LLMClientFactory.create("openai", "gpt-4o", 0.7f);

 // With max tokens
 LLMClient client = LLMClientFactory.create("anthropic", "claude-sonnet-4-20250514", 0.7f, 4096);

 // From RoleSpec
 RoleSpec metadata = RoleSpecReader.read(MyRole.class);
 LLMClient client = LLMClientFactory.fromSpec(metadata);

Environment Variables

Each provider reads its configuration from environment variables:

OLLAMA_BASE_URL, OLLAMA_API_KEY
OPENAI_API_KEY, OPENAI_BASE_URL
ANTHROPIC_API_KEY
GEMINI_API_KEY
GROQ_API_KEY
MISTRAL_API_KEY
COHERE_API_KEY
OPENROUTER_API_KEY
AZURE_OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT
AWS_REGION (Bedrock, default: us-east-1)
HUGGINGFACE_API_KEY
ZHIPU_API_KEY
MINIMAX_API_KEY, MINIMAX_GROUP_ID
CEREBRAS_API_KEY, CEREBRAS_BASE_URL (defaults to api.cerebras.ai/v1)
NVIDIA_API_KEY, NVIDIA_BASE_URL (defaults to integrate.api.nvidia.com)
DEEPINFRA_API_KEY, DEEPINFRA_BASE_URL (defaults to api.deepinfra.com/v1/openai)
FIREWORKS_API_KEY, FIREWORKS_BASE_URL (defaults to api.fireworks.ai/inference/v1)
TOGETHER_API_KEY, TOGETHER_BASE_URL (defaults to api.together.xyz/v1)
XAI_API_KEY, XAI_BASE_URL (defaults to api.x.ai/v1)
WATSONX_API_KEY, WATSONX_PROJECT_ID, WATSONX_BASE_URL (defaults to us-south.ml.cloud.ibm.com)
VERTEX_AI_API_KEY, VERTEX_AI_PROJECT_ID, VERTEX_AI_LOCATION (default us-central1), VERTEX_AI_BASE_URL
REPLICATE_API_KEY (Replicate marketplace; Replicate's docs call this REPLICATE_API_TOKEN)
DEEPSEEK_API_KEY (DeepSeek V3 chat + R1 reasoning)
LMSTUDIO_BASE_URL, LMSTUDIO_API_KEY (both optional; defaults to localhost:1234/v1)
PERPLEXITY_API_KEY (search-augmented Sonar family)
DATABRICKS_API_KEY, DATABRICKS_BASE_URL (workspace-scoped; no sensible default)
DASHSCOPE_API_KEY, DASHSCOPE_BASE_URL (defaults to dashscope-intl.aliyuncs.com/compatible-mode/v1)
LLAMACPP_BASE_URL, LLAMACPP_API_KEY (both optional; defaults to localhost:8080/v1)
VLLM_BASE_URL, VLLM_API_KEY (both optional; defaults to localhost:8000/v1)
HUNYUAN_API_KEY (Tencent Hunyuan, base URL defaults to api.hunyuan.cloud.tencent.com/v1)
YI_API_KEY (01.AI Yi, base URL defaults to api.lingyiwanwu.com/v1)

Author:

TnsAI Team

See Also:

Method Summary

Modifier and Type

Method

Description

static com.tnsai.llm.LLMClient

create(String provider, String model, float temperature)

Creates an LLM client from the given provider, model, and temperature.

static com.tnsai.llm.LLMClient

create(String provider, String model, float temperature, Integer maxTokens)

Creates an LLM client from the given configuration.

static com.tnsai.llm.LLMClient

create(String provider, String model, float temperature, Integer maxTokens, Float topP)

Creates an LLM client from the given configuration with topP support.

static com.tnsai.llm.LLMClient

create(String provider, String model, float temperature, Integer maxTokens, Float topP, LLMClientSettings settings)

Creates an LLM client honoring Phase 1 LLMClientSettings (endpoint, apiKeyEnv, timeoutMs, penalties).

static com.tnsai.llm.LLMClient

fromAnnotation(Class<?> clazz)

Creates an LLM client directly from a class's @RoleSpec annotation.

static com.tnsai.llm.LLMClient

fromSpec(com.tnsai.metadata.RoleSpecReader.RoleSpec metadata)

Creates an LLM client from role metadata.

static com.tnsai.llm.LLMClient

fromSpecSource(com.tnsai.llm.LLMSpecSource config, String provider)

Creates an LLM client from an LLMSpecSource.

Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details
- create
  
  public static com.tnsai.llm.LLMClient create(String provider, String model, float temperature)
  
  Creates an LLM client from the given provider, model, and temperature.
  
  Parameters:
  
  provider - the provider name (e.g., "ollama", "openai", "anthropic")
  
  model - the model name
  
  temperature - the sampling temperature (0.0-2.0)
  
  Returns:
  
  the created LLM client
  
  Throws:
  
  IllegalArgumentException - if provider is unknown or model is empty
- create
  
  public static com.tnsai.llm.LLMClient create(String provider, String model, float temperature, Integer maxTokens)
  
  Creates an LLM client from the given configuration.
  
  Parameters:
  
  provider - the provider name
  
  model - the model name
  
  temperature - the sampling temperature
  
  maxTokens - optional max tokens (null for provider default)
  
  Returns:
  
  the created LLM client
  
  Throws:
  
  IllegalArgumentException - if provider is unknown or model is empty
- create
  
  public static com.tnsai.llm.LLMClient create(String provider, String model, float temperature, Integer maxTokens, Float topP)
  
  Creates an LLM client from the given configuration with topP support.
  
  Parameters:
  
  provider - the provider name
  
  model - the model name
  
  temperature - the sampling temperature
  
  maxTokens - optional max tokens (null for provider default)
  
  topP - optional top-p nucleus sampling (null for provider default, reserved for future use)
  
  Returns:
  
  the created LLM client
  
  Throws:
  
  IllegalArgumentException - if provider is unknown or model is empty
- create
  
  public static com.tnsai.llm.LLMClient create(String provider, String model, float temperature, Integer maxTokens, Float topP, LLMClientSettings settings)
  
  Creates an LLM client honoring Phase 1 LLMClientSettings (endpoint, apiKeyEnv, timeoutMs, penalties). OpenAI and Ollama apply them; other providers fail loud if any setting is non-default.
- fromSpec
  
  public static com.tnsai.llm.LLMClient fromSpec(com.tnsai.metadata.RoleSpecReader.RoleSpec metadata)
  
  Creates an LLM client from role metadata.
  This method extracts LLM configuration from the metadata and creates the appropriate client. Returns null if no LLM is configured.
  
  Parameters:
  
  metadata - the role metadata
  
  Returns:
  
  the created LLM client, or null if no LLM configured
- fromSpecSource
  
  public static com.tnsai.llm.LLMClient fromSpecSource(com.tnsai.llm.LLMSpecSource config, String provider)
  
  Creates an LLM client from an LLMSpecSource.
  This method is useful for integration with LLMResolver.
  
  Parameters:
  
  config - the LLM config source
  
  provider - the provider name (config doesn't include provider)
  
  Returns:
  
  the created LLM client
- fromAnnotation
  
  public static com.tnsai.llm.LLMClient fromAnnotation(Class<?> clazz)
  Creates an LLM client directly from a class's @RoleSpec annotation.
  This is the simplest way to get annotation-driven LLM initialization for any class, regardless of what it extends.
  Usage
  
  // In your role constructor: this.llmClient = LLMClientFactory.fromAnnotation(getClass()); // Or for any annotated class: LLMClient client = LLMClientFactory.fromAnnotation(MyAnnotatedClass.class);
  Parameters:
  
  clazz - the class with @RoleSpec annotation
  
  Returns:
  
  the created LLM client, or null if no LLM configured

Class LLMClientFactory

Supported Providers

Usage with @LLMSpec Annotation

Programmatic Usage

Environment Variables

Method Summary

Methods inherited from class java.lang.Object

Method Details

create

create

create

create

fromSpec

fromSpecSource

fromAnnotation

Usage