Package com.tnsai.llm.observability

Class CapturingLLMClient

java.lang.Object

com.tnsai.llm.observability.CapturingLLMClient

All Implemented Interfaces:

com.tnsai.llm.LLMClient

public final class CapturingLLMClient
extends Object
implements com.tnsai.llm.LLMClient

Decorator that wraps any LLMClient and emits one LLMCallLog per call to a configured LLMCallPublisher (issue #79).

Captured per call:

• callId (UUID), startedAt / completedAt / elapsed
• provider + model (from LLMClient.getProviderName() + LLMClient.getModel())
• LLMPrompt — message, system prompt, history, parameters
• ToolSurface via ToolSurfaceHasher
• LLMResponse — content + tool calls (response side)
• TokenUsage — pulled from ChatResponse usage fields when the provider populated them; zero otherwise (the cost will also be zero, which is a meaningful "no usage data" signal)
• CostEstimate — via injected LLMPricingRegistry
• EventContext — current thread's AgentContext.currentOptional(), falling back to EventContext.empty()
• ErrorInfo — when the underlying call throws (re-throws after capture so caller sees the original exception)
• StreamMetrics for streaming calls — TTFT + chunk count; inter-chunk percentiles are p50 == p99 == zero in this PR (a single-pass percentile is doable but adds a sort per call, which fights the <2% perf budget; ship histogram-friendly counts + TTFT now, full percentiles in a follow-up)

Opt-in by construction: the framework default is no capture. Consumers wrap their LLMClient themselves (e.g. via an AgentBuilder convenience method) when they want the captured stream.

Multimodal chat(List<ContentPart> ...) and tool-aware streaming via streamChatWithSpec are passed through to the delegate without capture in this PR — those code paths are smaller in production usage and would double the surface area of this decorator before we have integration coverage. They route through the delegate cleanly so callers don't lose functionality.

Since:

0.9.0 (issue #79)

Constructor Summary

Constructors

Constructor	Description
CapturingLLMClient(com.tnsai.llm.LLMClient delegate, LLMPricingRegistry pricing, LLMCallPublisher publisher)
CapturingLLMClient(com.tnsai.llm.LLMClient delegate, LLMPricingRegistry pricing, LLMCallPublisher publisher, String endpoint)

Method Summary

Modifier and Type	Method	Description
com.tnsai.llm.ChatResponse	chat(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
com.tnsai.llm.ChatResponse	chat(List<com.tnsai.models.chat.ContentPart> messageParts, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
com.tnsai.llm.LLMCapabilities	getCapabilities()
com.tnsai.llm.LLMClient	getDelegate()
Optional<Integer>	getMaxTokens()
String	getModel()
String	getProviderName()
float	getTemperature()
Optional<Float>	getTopP()
Stream<String>	streamChat(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.tnsai.llm.LLMClient

chat, chat, chat, chat, listModels, streamChat, streamChat, streamChatWithHandler, streamChatWithHandler, streamChatWithSpec, streamChatWithSpec

Constructor Details

CapturingLLMClient

public CapturingLLMClient(com.tnsai.llm.LLMClient delegate,
 LLMPricingRegistry pricing,
 LLMCallPublisher publisher)

Parameters:

delegate - the underlying LLM client (required)

pricing - pricing registry; defaults to JsonLLMPricingRegistry.defaultRegistry() when null

publisher - publisher to receive LLMCallLogs; defaults to LLMCallPublisher.NOOP when null

CapturingLLMClient

public CapturingLLMClient(com.tnsai.llm.LLMClient delegate,
 LLMPricingRegistry pricing,
 LLMCallPublisher publisher,
 String endpoint)

Parameters:

endpoint - optional endpoint URL surfaced in LLMCallLog.endpoint(); useful when wrapping clients that don't expose the URL they're hitting. Pass empty string when unknown.

Method Details

getDelegate

public com.tnsai.llm.LLMClient getDelegate()

getModel

public String getModel()

Specified by:

getModel in interface com.tnsai.llm.LLMClient

getTemperature

public float getTemperature()

Specified by:

getTemperature in interface com.tnsai.llm.LLMClient

getTopP

public Optional<Float> getTopP()

Specified by:

getTopP in interface com.tnsai.llm.LLMClient

getMaxTokens

public Optional<Integer> getMaxTokens()

Specified by:

getMaxTokens in interface com.tnsai.llm.LLMClient

getCapabilities

public com.tnsai.llm.LLMCapabilities getCapabilities()

Specified by:

getCapabilities in interface com.tnsai.llm.LLMClient

getProviderName

public String getProviderName()

Specified by:

getProviderName in interface com.tnsai.llm.LLMClient

chat

public com.tnsai.llm.ChatResponse chat(String message,
 Optional<String> systemPrompt,
 Optional<List<Map<String,Object>>> history,
 Optional<List<Map<String,Object>>> tools)

Specified by:

chat in interface com.tnsai.llm.LLMClient

chat

public com.tnsai.llm.ChatResponse chat(List<com.tnsai.models.chat.ContentPart> messageParts,
 Optional<String> systemPrompt,
 Optional<List<Map<String,Object>>> history,
 Optional<List<Map<String,Object>>> tools)

Specified by:

chat in interface com.tnsai.llm.LLMClient

streamChat

public Stream<String> streamChat(String message,
 Optional<String> systemPrompt,
 Optional<List<Map<String,Object>>> history,
 Optional<List<Map<String,Object>>> tools)

Specified by:

streamChat in interface com.tnsai.llm.LLMClient