Package com.tnsai.llm.routing

Class CapabilityRouter

java.lang.Object
com.tnsai.llm.routing.CapabilityRouter
All Implemented Interfaces:
com.tnsai.llm.LLMClient, LLMRouter
public class CapabilityRouter extends Object implements LLMRouter
Routes LLM requests based on required capabilities.

CapabilityRouter automatically selects the appropriate model based on the capabilities needed for each request. This is useful for multi-modal applications where some requests need vision and others don't.

Usage Example


 // Create router with different capability profiles
 CapabilityRouter router = CapabilityRouter.builder()
     .addClient(new OpenAIClient("gpt-4o"))        // Vision + Tools
     .addClient(new GroqClient("llama-3.3-70b"))  // Tools only
     .addClient(new OllamaClient("llama3.2"))     // Basic text
     .build();

 // Select vision-capable model for image inputs
 LLMClient visionClient = router.selectByCapability(cap -> cap.supportsVision())
     .orElseThrow();

 // Select model with function calling for tool use
 LLMClient toolClient = router.selectByCapability(cap -> cap.supportsFunctionCalling())
     .orElseThrow();

Default Capability Selection


 // Set default capability requirement
 CapabilityRouter router = CapabilityRouter.builder()
     .addClient(new OpenAIClient("gpt-4o"))
     .addClient(new GroqClient("llama-3.3-70b"))
     .defaultRequirement(cap -> cap.supportsFunctionCalling())
     .build();

 // All requests go to function-calling capable models
 router.chat("Use the search tool");
See Also:

  • Method Details

    • of

      public static CapabilityRouter of(com.tnsai.llm.LLMClient... clients)
      Creates a CapabilityRouter with the given clients.

    • builder

      public static CapabilityRouter.Builder builder()
      Returns a builder for configuring the router.

    • getClients

      public List<com.tnsai.llm.LLMClient> getClients()
      Description copied from interface: LLMRouter
      Returns all registered LLM clients.
      Specified by:
      getClients in interface LLMRouter

    • getPrimaryClient

      public com.tnsai.llm.LLMClient getPrimaryClient()
      Description copied from interface: LLMRouter
      Returns the currently selected/primary client.
      Specified by:
      getPrimaryClient in interface LLMRouter

    • selectByCapability

      public Optional<com.tnsai.llm.LLMClient> selectByCapability(Predicate<com.tnsai.llm.LLMCapabilities> filter)
      Description copied from interface: LLMRouter
      Selects a client that matches the given capability predicate.
      Specified by:
      selectByCapability in interface LLMRouter
      Parameters:
      filter - Predicate to filter by capabilities
      Returns:
      The first matching client, or empty if none match

    • selectVisionCapable

      public Optional<com.tnsai.llm.LLMClient> selectVisionCapable()
      Selects the best client for vision tasks.

    • selectToolCapable

      public Optional<com.tnsai.llm.LLMClient> selectToolCapable()
      Selects the best client for tool/function calling.

    • selectStructuredOutputCapable

      public Optional<com.tnsai.llm.LLMClient> selectStructuredOutputCapable()
      Selects the best client for structured output (JSON mode).

    • selectWithMinContext

      public Optional<com.tnsai.llm.LLMClient> selectWithMinContext(int minTokens)
      Selects a client with minimum context window size.

    • getModel

      public String getModel()
      Specified by:
      getModel in interface com.tnsai.llm.LLMClient

    • getTemperature

      public float getTemperature()
      Specified by:
      getTemperature in interface com.tnsai.llm.LLMClient

    • getMaxTokens

      public Optional<Integer> getMaxTokens()
      Specified by:
      getMaxTokens in interface com.tnsai.llm.LLMClient

    • getCapabilities

      public com.tnsai.llm.LLMCapabilities getCapabilities()
      Specified by:
      getCapabilities in interface com.tnsai.llm.LLMClient

    • chat

      public com.tnsai.llm.ChatResponse chat(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
      Description copied from interface: LLMRouter
      Routes a chat request to the appropriate client.

      The routing decision depends on the implementation:

      • FallbackRouter: Uses primary, falls back on error
      • CostBasedRouter: Selects cheapest viable option
      • LatencyBasedRouter: Selects fastest option
      Specified by:
      chat in interface com.tnsai.llm.LLMClient
      Specified by:
      chat in interface LLMRouter
      Parameters:
      message - The user message
      systemPrompt - Optional system prompt
      history - Optional conversation history
      tools - Optional tools
      Returns:
      The chat response from the selected client

    • streamChat

      public Stream<String> streamChat(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
      Specified by:
      streamChat in interface com.tnsai.llm.LLMClient

    • streamChatWithSpec

      public Stream<com.tnsai.llm.ChatChunk> streamChatWithSpec(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
      Specified by:
      streamChatWithSpec in interface com.tnsai.llm.LLMClient

    • getStats

      public LLMRouter.RoutingStats getStats()
      Description copied from interface: LLMRouter
      Returns routing statistics (calls per provider, failures, latencies).
      Specified by:
      getStats in interface LLMRouter

    • resetStats

      public void resetStats()
      Description copied from interface: LLMRouter
      Resets routing statistics.
      Specified by:
      resetStats in interface LLMRouter