Package com.tnsai.agents.execution

Class LLMOrchestrator

java.lang.Object
com.tnsai.agents.execution.LLMOrchestrator
public class LLMOrchestrator extends Object
Orchestrates LLM communication for agents.

This class is extracted from Agent to provide a focused component responsible for all LLM-related operations:

  • Invoking LLM with various configurations
  • Processing LLM responses (text and tool calls)
  • Managing conversation history in memory store
  • Supporting streaming responses
  • Handling spinner display during LLM calls

Design Principles

  • Single responsibility: LLM communication only
  • Stateless operations: receives context via method parameters
  • Callback-based: delegates tool execution back to agent

Usage


 LLMOrchestrator orchestrator = new LLMOrchestrator(
     agentName,
     llmClient,
     memoryStore
 );

 // Specure callbacks
 orchestrator.setToolCallHandler(chatResponse ->
     toolExecutor.executeToolCalls(chatResponse, true));
 orchestrator.setSystemPromptSupplier(() -> generateSystemPrompt());
 orchestrator.setToolsSupplier(() -> getAvailableTools());

 // Invoke LLM
 String response = orchestrator.chat("Hello", true);

 // Streaming
 orchestrator.streamChat("Tell me a story")
     .forEach(System.out::print);
See Also:

  • Constructor Details

    • LLMOrchestrator

      public LLMOrchestrator(String agentName, LLMClient llm, MemoryStore memoryStore)
      Creates a new LLMOrchestrator.
      Parameters:
      agentName - the agent name for logging and tracing
      llm - the LLM client to use
      memoryStore - the memory store for conversation history

  • Method Details

    • setMaxContextTokens

      public LLMOrchestrator setMaxContextTokens(Integer maxTokens)
      Sets the maximum context token limit. When set, conversation history is automatically pruned to stay within this limit.
      Parameters:
      maxTokens - maximum tokens (null to disable pruning)
      Returns:
      this for chaining

    • setSystemPromptSupplier

      public LLMOrchestrator setSystemPromptSupplier(Supplier<String> supplier)
      Sets the system prompt supplier.
      Parameters:
      supplier - function that returns the current system prompt
      Returns:
      this for chaining

    • setToolsSupplier

      public LLMOrchestrator setToolsSupplier(Supplier<List<Map<String,Object>>> supplier)
      Sets the tools supplier.
      Parameters:
      supplier - function that returns available tools for the LLM
      Returns:
      this for chaining

    • setToolCallHandler

      public LLMOrchestrator setToolCallHandler(LLMOrchestrator.ToolCallHandler handler)
      Sets the tool call handler.
      Parameters:
      handler - function to execute tool calls
      Returns:
      this for chaining

    • setSpinnerEnabled

      public LLMOrchestrator setSpinnerEnabled(boolean enabled)
      Enables or disables spinner during LLM calls.
      Parameters:
      enabled - true to show spinner
      Returns:
      this for chaining

    • chat

      public String chat(String message, boolean useHistory)
      Sends a message to the LLM and returns the response.

      This method:

      1. Adds the user message to history (if useHistory is true)
      2. Prunes history if token limit is set
      3. Invokes the LLM with system prompt, history, and tools
      4. Processes the response (handles tool calls if configured)
      5. Adds the assistant response to history
      Parameters:
      message - the user message
      useHistory - whether to include conversation history
      Returns:
      the LLM response

    • chat

      public String chat(String message, boolean useHistory, boolean useTools, boolean trace)
      Sends a message with full configuration options.
      Parameters:
      message - the user message
      useHistory - whether to include conversation history
      useTools - whether to provide tools to the LLM
      trace - whether to trace the request/response
      Returns:
      the LLM response

    • chatInternal

      public String chatInternal(String message, boolean useHistory)
      Internal chat without tool execution. Prevents recursion when called from tool execution handlers.
      Parameters:
      message - the message
      useHistory - whether to use history
      Returns:
      the response

    • streamChat

      public Stream<String> streamChat(String message)
      Sends a message and returns a streaming response.

      Unlike chat(String, boolean), this method returns a Stream that yields response tokens as they are generated.

      Note: Tool calling is not supported in streaming mode.

      Parameters:
      message - the user message
      Returns:
      stream of response tokens

    • streamChatWithConsumer

      public void streamChatWithConsumer(String message, Consumer<String> tokenConsumer)
      Sends a message with streaming and custom event handling.
      Parameters:
      message - the user message
      tokenConsumer - consumer for each token

    • getHistory

      public List<Map<String,Object>> getHistory()
      Returns the current conversation history.
      Returns:
      list of message maps

    • clearHistory

      public void clearHistory()
      Clears the conversation history.

    • getLLM

      public LLMClient getLLM()
      Returns the configured LLM client.
      Returns:
      the LLM client

    • getMemoryStore

      public MemoryStore getMemoryStore()
      Returns the memory store.
      Returns:
      the memory store

    • getAgentName

      public String getAgentName()
      Returns the agent name.
      Returns:
      the agent name