Package com.tnsai.eval.hooks

Class EvalContext

java.lang.Object
com.tnsai.eval.hooks.EvalContext
public final class EvalContext extends Object
Context for collecting and managing evaluation metrics during agent execution.

EvalContext is a thread-safe container that aggregates metrics from multiple sources (hooks, annotations, manual recording) and provides access to evaluation results.

Features

  • Thread-safe metric recording
  • Real-time metric streaming
  • Hierarchical metric organization
  • Statistical aggregation
  • Export to various formats

Usage


 EvalContext ctx = EvalContext.create("session-123", "research-agent");

 // Record metrics
 ctx.recordMetric("latency", 150);
 ctx.recordMetric("accuracy", 0.95);
 ctx.incrementCounter("tool_calls");

 // Get statistics
 MetricStats stats = ctx.getStats("latency");
 System.out.println("Avg latency: " + stats.getAverage());

 // Export
 Map<String, Object> report = ctx.toMap();
Author:
TnsAI Team
See Also:

  • Method Details

    • create

      public static EvalContext create(String sessionId, String agentId)
      Creates a new evaluation context.
      Parameters:
      sessionId - session identifier
      agentId - agent identifier
      Returns:
      new context

    • recordMetric

      public void recordMetric(String name, double value)
      Records a numeric metric value.
      Parameters:
      name - metric name
      value - metric value

    • recordMetric

      public void recordMetric(String name, double value, Map<String,String> tags)
      Records a metric with tags.
      Parameters:
      name - metric name
      value - metric value
      tags - metric tags

    • incrementCounter

      public void incrementCounter(String name)
      Increments a counter by 1.
      Parameters:
      name - counter name

    • incrementCounter

      public void incrementCounter(String name, long amount)
      Increments a counter by a specific amount.
      Parameters:
      name - counter name
      amount - amount to increment

    • setSpec

      public void setSpec(String key, Object value)
      Sets metadata.
      Parameters:
      key - metadata key
      value - metadata value

    • getSpec

      public Object getSpec(String key)
      Gets metadata.
      Parameters:
      key - metadata key
      Returns:
      metadata value or null

    • getMetricValues

      public List<Double> getMetricValues(String name)
      Gets all values for a metric.
      Parameters:
      name - metric name
      Returns:
      list of values

    • getCounter

      public long getCounter(String name)
      Gets counter value.
      Parameters:
      name - counter name
      Returns:
      counter value

    • getStats

      public EvalContext.MetricStats getStats(String name)
      Gets statistics for a metric.
      Parameters:
      name - metric name
      Returns:
      statistics

    • getMetricNames

      public Set<String> getMetricNames()
      Gets all metric names.
      Returns:
      set of metric names

    • getCounterNames

      public Set<String> getCounterNames()
      Gets all counter names.
      Returns:
      set of counter names

    • addListener

      public void addListener(Consumer<EvalContext.MetricEvent> listener)
      Adds a real-time metric listener.
      Parameters:
      listener - listener to add

    • removeListener

      public void removeListener(Consumer<EvalContext.MetricEvent> listener)
      Removes a metric listener.
      Parameters:
      listener - listener to remove

    • complete

      public void complete(String status)
      Marks the context as completed.
      Parameters:
      status - final status

    • getSessionId

      public String getSessionId()
      Gets session ID.
      Returns:
      session ID

    • getAgentId

      public String getAgentId()
      Gets agent ID.
      Returns:
      agent ID

    • getStartTime

      public Instant getStartTime()
      Gets start time.
      Returns:
      start time

    • getEndTime

      public Instant getEndTime()
      Gets end time.
      Returns:
      end time or null if not completed

    • getDurationMs

      public long getDurationMs()
      Gets duration in milliseconds.
      Returns:
      duration or -1 if not completed

    • getStatus

      public String getStatus()
      Gets status.
      Returns:
      status

    • getEvents

      public List<EvalContext.MetricEvent> getEvents()
      Gets all events.
      Returns:
      list of events

    • toMap

      public Map<String,Object> toMap()
      Converts context to a map for serialization.
      Returns:
      map representation