Package com.tnsai.llm.observability

Interface LLMObserver

All Known Implementing Classes:
LLMMetrics, ObservableLLMClient.CompositeObserver, PromptLogger
public interface LLMObserver
Observer interface for LLM request/response monitoring.

LLMObserver enables non-invasive monitoring of all LLM interactions. Implement this interface to add custom logging, metrics collection, or audit trails.

Usage Example


 // Create a logging observer
 LLMObserver loggingObserver = new LLMObserver() {
     @Override
     public void onRequest(LLMClient client, String message,
                          Optional<String> systemPrompt,
                          Optional<List<Map<String, Object>>> history,
                          Optional<List<Map<String, Object>>> tools) {
         logger.info("LLM Request to {}: {}", client.getModel(), message);
     }

     @Override
     public void onResponse(LLMClient client, ChatResponse response, long latencyMs) {
         logger.info("LLM Response from {} ({}ms): {}",
             client.getModel(), latencyMs, response.getContent().substring(0, 100));
     }

     @Override
     public void onError(LLMClient client, Exception error, long latencyMs) {
         logger.error("LLM Error from {} ({}ms): {}",
             client.getModel(), latencyMs, error.getMessage());
     }
 };

 // Wrap client with observer
 LLMClient observedClient = new ObservableLLMClient(client, loggingObserver);

Multiple Observers


 // Combine multiple observers
 LLMObserver combined = CompositeObserver.of(
     loggingObserver,
     metricsObserver,
     auditObserver
 );
See Also:

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final LLMObserver
    NOOP
    No-op observer that does nothing.

  • Method Summary

    Modifier and Type
    Method
    Description
    default void
    onError(com.tnsai.llm.LLMClient client, Exception error, long latencyMs)
    Called when an error occurs.
    default void
    onRequest(com.tnsai.llm.LLMClient client, String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
    Called before sending a request to the LLM.
    default void
    onResponse(com.tnsai.llm.LLMClient client, com.tnsai.llm.ChatResponse response, long latencyMs)
    Called after receiving a successful response.
    default void
    onStreamChunk(com.tnsai.llm.LLMClient client, String chunk, int chunkIndex)
    Called for each streaming chunk received.
    default void
    onStreamComplete(com.tnsai.llm.LLMClient client, int totalChunks, long latencyMs)
    Called when streaming completes.
    default void
    onStreamError(com.tnsai.llm.LLMClient client, Exception error, int chunksReceived, long latencyMs)
    Called when streaming encounters an error.

  • Field Details

    • NOOP

      static final LLMObserver NOOP
      No-op observer that does nothing.

  • Method Details

    • onRequest

      default void onRequest(com.tnsai.llm.LLMClient client, String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
      Called before sending a request to the LLM.
      Parameters:
      client - The LLM client being used
      message - The user message
      systemPrompt - Optional system prompt
      history - Optional conversation history
      tools - Optional tools

    • onResponse

      default void onResponse(com.tnsai.llm.LLMClient client, com.tnsai.llm.ChatResponse response, long latencyMs)
      Called after receiving a successful response.
      Parameters:
      client - The LLM client that was used
      response - The response received
      latencyMs - Time taken in milliseconds

    • onError

      default void onError(com.tnsai.llm.LLMClient client, Exception error, long latencyMs)
      Called when an error occurs.
      Parameters:
      client - The LLM client that was used
      error - The exception that occurred
      latencyMs - Time taken before error in milliseconds

    • onStreamChunk

      default void onStreamChunk(com.tnsai.llm.LLMClient client, String chunk, int chunkIndex)
      Called for each streaming chunk received.
      Parameters:
      client - The LLM client being used
      chunk - The content chunk
      chunkIndex - The index of this chunk (0-based)

    • onStreamComplete

      default void onStreamComplete(com.tnsai.llm.LLMClient client, int totalChunks, long latencyMs)
      Called when streaming completes.
      Parameters:
      client - The LLM client that was used
      totalChunks - Total number of chunks received
      latencyMs - Total time taken in milliseconds

    • onStreamError

      default void onStreamError(com.tnsai.llm.LLMClient client, Exception error, int chunksReceived, long latencyMs)
      Called when streaming encounters an error.
      Parameters:
      client - The LLM client that was used
      error - The exception that occurred
      chunksReceived - Number of chunks received before error
      latencyMs - Time taken before error in milliseconds