Package com.tnsai.llm.observability

package com.tnsai.llm.observability
Observability components for LLM operations.

The observability package provides comprehensive monitoring, logging, and metrics collection for all LLM interactions.

Two parallel observation channels

This package ships two observer surfaces that target different consumers — both are first-class, neither deprecates the other:

1. Raw-string LLMObserver (legacy / lightweight)

ClassPurpose
LLMObserver Observer interface for request/response monitoring
ObservableLLMClient Wrapper that adds observability to any LLMClient
LLMMetrics Collects metrics: requests, tokens, latency, cost
PromptLogger Structured logging with PII filtering

2. Typed LLMCallLog channel (issue #79)

ClassPurpose
LLMCallLog One immutable record per LLM call — provider, model, prompt, response, token usage, USD cost, error info, stream metrics, EventContext
LLMCallPublisher SPI for receiving captured logs (LangFuse / Helicone bridges, Prometheus derivation, cost dashboards)
Slf4jLLMCallPublisher Default publisher — one structured SLF4J line per call (no raw prompt/response text — that's behind redaction SPI #80)
CapturingLLMClient Decorator wrapping any LLMClient — opt-in capture, zero overhead when not wrapped
LLMPricingRegistry SPI for converting TokenUsage into a USD CostEstimate
JsonLLMPricingRegistry Loads rate cards from classpath JSON; ships /pricing/2026-05.json with openai / anthropic / ollama rates out of the box
ToolSurfaceHasher Builds ToolSurface with a stable SHA-256 hash so identical tool sets across calls correlate (prompt-cache friendly)

Capturing channel — quick start


 LLMClient base = new OpenAIClient("gpt-4o");
 LLMClient captured = new CapturingLLMClient(
         base,
         JsonLLMPricingRegistry.defaultRegistry(),
         new Slf4jLLMCallPublisher());

 captured.chat("hi", Optional.empty(), Optional.empty(), Optional.empty());
 // → one INFO line: "llm.call provider=openai model=gpt-4o elapsedMs=420 ..."

Quick Start


 // Create observers
 LLMMetrics metrics = new LLMMetrics();
 PromptLogger logger = PromptLogger.withPIIFiltering();

 // Wrap client with observability
 LLMClient baseClient = new OpenAIClient("gpt-4o");
 LLMClient observedClient = new ObservableLLMClient(baseClient, metrics, logger);

 // Use normally - all calls are tracked
 observedClient.chat("Hello!");

 // Get metrics report
 LLMMetrics.Report report = metrics.getReport();
 System.out.println("Total requests: " + report.totalRequests());
 System.out.println("Avg latency: " + report.avgLatencyMs() + "ms");
 System.out.println("Estimated cost: $" + report.totalEstimatedCost());

Metrics Tracked

  • Request/response counts per provider
  • Token usage (estimated) per provider
  • Latency statistics (avg, p50, p95, p99)
  • Error rates per provider
  • Cost estimates based on token usage
  • Streaming chunk counts

PII Filtering

The PromptLogger can automatically redact sensitive information:

  • Email addresses → [REDACTED_EMAIL]
  • Phone numbers → [REDACTED_PHONE]
  • Credit cards → [REDACTED_CC]
  • SSN → [REDACTED_SSN]
  • API keys → [REDACTED_KEY]
See Also: