Package com.tnsai.llm.routing

Interface LLMRouter

All Superinterfaces:

com.tnsai.llm.LLMClient

All Known Implementing Classes:

CapabilityRouter, CostBasedRouter, FallbackRouter, LatencyBasedRouter, RoundRobinRouter, TaskBasedRouter

public interface LLMRouter
extends com.tnsai.llm.LLMClient

Routes LLM requests to appropriate providers based on configurable strategies.

LLMRouter enables intelligent model selection and provides resilience through automatic fallback when providers fail. This is critical for production systems where reliability and cost optimization are important.

Routing Strategies

Strategy	Implementation	Use Case
Fallback	FallbackRouter	Reliability - try alternatives on failure
Cost-based	CostBasedRouter	Budget optimization
Latency-based	LatencyBasedRouter	Performance-critical apps
Capability	CapabilityRouter	Feature-based selection
Round-robin	RoundRobinRouter	Load distribution

Usage Example

 // Create router with fallback
 LLMRouter router = FallbackRouter.of(
     new OpenAIClient("gpt-4o"),
     new AnthropicClient("claude-sonnet-4"),
     new GroqClient("llama-3.3-70b")
 );

 // Use like a normal client
 ChatResponse response = router.chat("Hello!");

 // If GPT-4o fails, automatically tries Claude, then Groq
 

Cost Optimization

 // Route simple queries to cheap models
 LLMRouter router = CostBasedRouter.builder()
     .addProvider(new OpenAIClient("gpt-4o-mini"))  // Cheapest
     .addProvider(new OpenAIClient("gpt-4o"))       // Premium
     .build();

 // Simple queries go to gpt-4o-mini
 router.chat("What is 2+2?");

 // Complex queries might route to gpt-4o
 router.chat("Explain quantum computing in detail");
 

Capability-based Selection

 // Auto-select vision-capable model
 LLMRouter router = CapabilityRouter.builder()
     .addProvider(new OpenAIClient("gpt-4o"))      // Has vision
     .addProvider(new GroqClient("llama-3.3-70b")) // No vision
     .build();

 // Automatically selects GPT-4o for image inputs
 router.selectByCapability(cap -> cap.supportsVision());
 

See Also:

• FallbackRouter
• CostBasedRouter
• LatencyBasedRouter
• CapabilityRouter

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static final record	LLMRouter.RoutingStats	Records for routing statistics.

Method Summary

Modifier and Type	Method	Description
com.tnsai.llm.ChatResponse	chat(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)	Routes a chat request to the appropriate client.
List<com.tnsai.llm.LLMClient>	getClients()	Returns all registered LLM clients.
com.tnsai.llm.LLMClient	getPrimaryClient()	Returns the currently selected/primary client.
default LLMRouter.RoutingStats	getStats()	Returns routing statistics (calls per provider, failures, latencies).
default void	resetStats()	Resets routing statistics.
Optional<com.tnsai.llm.LLMClient>	selectByCapability(Predicate<com.tnsai.llm.LLMCapabilities> capabilityFilter)	Selects a client that matches the given capability predicate.

Methods inherited from interface com.tnsai.llm.LLMClient

chat, chat, chat, chat, chat, getCapabilities, getMaxTokens, getModel, getProviderName, getTemperature, getTopP, listModels, streamChat, streamChat, streamChat, streamChatWithHandler, streamChatWithHandler, streamChatWithSpec, streamChatWithSpec

Method Details

getClients

List<com.tnsai.llm.LLMClient> getClients()

Returns all registered LLM clients.

getPrimaryClient

com.tnsai.llm.LLMClient getPrimaryClient()

Returns the currently selected/primary client.

selectByCapability

Optional<com.tnsai.llm.LLMClient> selectByCapability(Predicate<com.tnsai.llm.LLMCapabilities> capabilityFilter)

Selects a client that matches the given capability predicate.

Parameters:

capabilityFilter - Predicate to filter by capabilities

Returns:

The first matching client, or empty if none match

chat

com.tnsai.llm.ChatResponse chat(String message,
 Optional<String> systemPrompt,
 Optional<List<Map<String,Object>>> history,
 Optional<List<Map<String,Object>>> tools)

Routes a chat request to the appropriate client.

The routing decision depends on the implementation:

• FallbackRouter: Uses primary, falls back on error
• CostBasedRouter: Selects cheapest viable option
• LatencyBasedRouter: Selects fastest option

Specified by:

chat in interface com.tnsai.llm.LLMClient

Parameters:

message - The user message

systemPrompt - Optional system prompt

history - Optional conversation history

tools - Optional tools

Returns:

The chat response from the selected client

getStats

default LLMRouter.RoutingStats getStats()

Returns routing statistics (calls per provider, failures, latencies).

resetStats

default void resetStats()

Resets routing statistics.