Package com.tnsai.llm.routing

Class TaskBasedRouter

java.lang.Object
com.tnsai.llm.routing.TaskBasedRouter
All Implemented Interfaces:
com.tnsai.llm.LLMClient, LLMRouter
public class TaskBasedRouter extends Object implements LLMRouter
Routes LLM requests to the most appropriate model based on task type.

TaskBasedRouter automatically classifies incoming prompts and routes them to specialized models. This enables cost optimization and quality improvement by using the right model for each task.

Key Benefits

  • Cost Savings: Simple tasks go to cheap models (up to 77% savings)
  • Quality: Complex tasks use specialized models (coding → Claude, math → o1)
  • Automatic: No manual model selection needed
  • Transparent: Full statistics on task routing

Usage Example


 // Create router with task-specific models
 TaskBasedRouter router = TaskBasedRouter.builder()
     .forTask(TaskType.CODING, new AnthropicClient("claude-sonnet-4"))
     .forTask(TaskType.CREATIVE, new OpenAIClient("gpt-4o"))
     .forTask(TaskType.MATH, new OpenAIClient("o1-preview"))
     .forTask(TaskType.FAST, new GroqClient("llama-3.3-70b"))
     .forTask(TaskType.VISION, new OpenAIClient("gpt-4o"))
     .defaultClient(new OpenAIClient("gpt-4o-mini"))
     .build();

 // Auto-routing based on prompt content
 router.chat("Write a Python function to sort a list");  // → Claude (CODING)
 router.chat("Solve: ∫x²dx");                            // → o1 (MATH)
 router.chat("Write a poem about autumn");               // → GPT-4o (CREATIVE)
 router.chat("What is 2+2?");                            // → GPT-4o-mini (FAST/default)

 // Check routing statistics
 TaskRoutingStats stats = router.getTaskStats();
 System.out.println("Requests per task: " + stats.requestsPerTask());
 System.out.println("Estimated savings: $" + stats.estimatedSavings());

Manual Override


 // Force a specific task type
 router.chatAs(TaskType.CODING, "Explain this concept");

 // Get the client for a specific task
 LLMClient codingClient = router.getClientForTask(TaskType.CODING);

With Fallback


 // Combine with FallbackRouter for reliability
 LLMRouter codingFallback = FallbackRouter.of(
     new AnthropicClient("claude-sonnet-4"),
     new OpenAIClient("gpt-4o")
 );

 TaskBasedRouter router = TaskBasedRouter.builder()
     .forTask(TaskType.CODING, codingFallback)  // With fallback
     .defaultClient(new GroqClient("llama-3.3-70b"))
     .build();
See Also:

  • Method Details

    • builder

      public static TaskBasedRouter.Builder builder()
      Creates a new builder for TaskBasedRouter.

    • getClientForTask

      public com.tnsai.llm.LLMClient getClientForTask(TaskType taskType)
      Returns the client assigned to a specific task type.
      Parameters:
      taskType - The task type
      Returns:
      The assigned client, or default client if not configured

    • classifyTask

      public TaskClassifier.ClassificationResult classifyTask(String message)
      Classifies the message and returns the detected task type.
      Parameters:
      message - The message to classify
      Returns:
      Classification result with task type and confidence

    • classifyTask

      public TaskClassifier.ClassificationResult classifyTask(String message, List<String> history)
      Classifies the message with conversation history context.
      Parameters:
      message - The message to classify
      history - Previous messages for context
      Returns:
      Classification result with task type and confidence

    • chatAs

      public com.tnsai.llm.ChatResponse chatAs(TaskType taskType, String message)
      Chat with explicit task type override.
      Parameters:
      taskType - The task type to use (bypasses classification)
      message - The message
      Returns:
      Chat response

    • chatAs

      public com.tnsai.llm.ChatResponse chatAs(TaskType taskType, String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
      Chat with explicit task type override and full parameters.

    • chat

      public com.tnsai.llm.ChatResponse chat(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
      Description copied from interface: LLMRouter
      Routes a chat request to the appropriate client.

      The routing decision depends on the implementation:

      • FallbackRouter: Uses primary, falls back on error
      • CostBasedRouter: Selects cheapest viable option
      • LatencyBasedRouter: Selects fastest option
      Specified by:
      chat in interface com.tnsai.llm.LLMClient
      Specified by:
      chat in interface LLMRouter
      Parameters:
      message - The user message
      systemPrompt - Optional system prompt
      history - Optional conversation history
      tools - Optional tools
      Returns:
      The chat response from the selected client

    • streamChat

      public Stream<String> streamChat(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
      Specified by:
      streamChat in interface com.tnsai.llm.LLMClient

    • streamChatWithSpec

      public Stream<com.tnsai.llm.ChatChunk> streamChatWithSpec(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
      Specified by:
      streamChatWithSpec in interface com.tnsai.llm.LLMClient

    • getClients

      public List<com.tnsai.llm.LLMClient> getClients()
      Description copied from interface: LLMRouter
      Returns all registered LLM clients.
      Specified by:
      getClients in interface LLMRouter

    • getPrimaryClient

      public com.tnsai.llm.LLMClient getPrimaryClient()
      Description copied from interface: LLMRouter
      Returns the currently selected/primary client.
      Specified by:
      getPrimaryClient in interface LLMRouter

    • selectByCapability

      public Optional<com.tnsai.llm.LLMClient> selectByCapability(Predicate<com.tnsai.llm.LLMCapabilities> filter)
      Description copied from interface: LLMRouter
      Selects a client that matches the given capability predicate.
      Specified by:
      selectByCapability in interface LLMRouter
      Parameters:
      filter - Predicate to filter by capabilities
      Returns:
      The first matching client, or empty if none match

    • getModel

      public String getModel()
      Specified by:
      getModel in interface com.tnsai.llm.LLMClient

    • getTemperature

      public float getTemperature()
      Specified by:
      getTemperature in interface com.tnsai.llm.LLMClient

    • getMaxTokens

      public Optional<Integer> getMaxTokens()
      Specified by:
      getMaxTokens in interface com.tnsai.llm.LLMClient

    • getCapabilities

      public com.tnsai.llm.LLMCapabilities getCapabilities()
      Specified by:
      getCapabilities in interface com.tnsai.llm.LLMClient

    • getTaskStats

      public TaskBasedRouter.TaskRoutingStats getTaskStats()
      Returns task-specific routing statistics.

    • getStats

      public LLMRouter.RoutingStats getStats()
      Description copied from interface: LLMRouter
      Returns routing statistics (calls per provider, failures, latencies).
      Specified by:
      getStats in interface LLMRouter

    • resetStats

      public void resetStats()
      Description copied from interface: LLMRouter
      Resets routing statistics.
      Specified by:
      resetStats in interface LLMRouter