Package com.tnsai.eval.hooks

Interface EvalHook

public interface EvalHook
Hook interface for integrating evaluation metrics into agent lifecycle.

EvalHook provides callback methods that are invoked at key points during agent execution, enabling metric collection without modifying agent code.

Lifecycle Events

 Agent Lifecycle:

   onAgentStart()
        │
        ▼
   onBeforeChat() ─────────────────────┐
        │                               │
        ▼                               │ (loop)
   onBeforeToolCall()                   │
        │                               │
        ▼                               │
   onAfterToolCall()                    │
        │                               │
        ▼                               │
   onAfterChat() ◄─────────────────────┘
        │
        ▼
   onPlanGenerated()
        │
        ▼
   onPlanStepExecuted() ──┐
        │                 │ (loop)
        ▼                 │
   onPlanCompleted() ◄────┘
        │
        ▼
   onGoalCompleted()
        │
        ▼
   onAgentStop()

Usage Example


 EvalHook metricsHook = new EvalHook() {
     @Override
     public void onAfterChat(EvalContext ctx, String response, long latencyMs) {
         ctx.recordMetric("latency", latencyMs);
         ctx.recordMetric("response_length", response.length());
     }

     @Override
     public void onGoalCompleted(EvalContext ctx, String goalId, boolean success) {
         ctx.recordMetric("goal_" + goalId, success ? 1.0 : 0.0);
     }
 };

 agent.addEvalHook(metricsHook);
Author:
TnsAI Team
See Also:

  • Method Details

    • onAgentStart

      default void onAgentStart(EvalContext ctx, String agentId, String sessionId)
      Called when agent starts a session.
      Parameters:
      ctx - evaluation context
      agentId - agent identifier
      sessionId - session identifier

    • onBeforeChat

      default void onBeforeChat(EvalContext ctx, String message)
      Called before processing a chat message.
      Parameters:
      ctx - evaluation context
      message - user message

    • onAfterChat

      default void onAfterChat(EvalContext ctx, String response, long latencyMs)
      Called after processing a chat message.
      Parameters:
      ctx - evaluation context
      response - agent response
      latencyMs - latency in milliseconds

    • onBeforeToolCall

      default void onBeforeToolCall(EvalContext ctx, String toolName, Map<String,Object> arguments)
      Called before executing a tool call.
      Parameters:
      ctx - evaluation context
      toolName - name of the tool
      arguments - tool arguments

    • onAfterToolCall

      default void onAfterToolCall(EvalContext ctx, String toolName, Object result, boolean success, long latencyMs)
      Called after executing a tool call.
      Parameters:
      ctx - evaluation context
      toolName - name of the tool
      result - tool result
      success - whether the call succeeded
      latencyMs - latency in milliseconds

    • onGoalCompleted

      default void onGoalCompleted(EvalContext ctx, String goalId, boolean success, Map<String,Object> details)
      Called when a goal is completed (success or failure).
      Parameters:
      ctx - evaluation context
      goalId - goal identifier
      success - whether goal was achieved
      details - additional details

    • onAgentStop

      default void onAgentStop(EvalContext ctx, String reason)
      Called when agent stops.
      Parameters:
      ctx - evaluation context
      reason - stop reason

    • onError

      default void onError(EvalContext ctx, Throwable error, String phase)
      Called when an error occurs.
      Parameters:
      ctx - evaluation context
      error - the error
      phase - which phase the error occurred in

    • onMemoryAccess

      default void onMemoryAccess(EvalContext ctx, String operation, String key, int resultCount, long latencyMs)
      Called when memory is accessed.
      Parameters:
      ctx - evaluation context
      operation - memory operation (read/write/search)
      key - memory key or query
      resultCount - number of results
      latencyMs - latency in milliseconds

    • onAgentCommunication

      default void onAgentCommunication(EvalContext ctx, String fromAgent, String toAgent, String messageType, long latencyMs)
      Called when inter-agent communication occurs.
      Parameters:
      ctx - evaluation context
      fromAgent - source agent
      toAgent - target agent
      messageType - type of message
      latencyMs - latency in milliseconds

    • onPlanGenerated

      default void onPlanGenerated(EvalContext ctx, String goalId, List<PlannerHandle.PlanStep> steps, long latencyMs)
      Called when a plan is generated for a goal.
      Parameters:
      ctx - evaluation context
      goalId - goal identifier (empty string for all-goals planning)
      steps - generated plan steps
      latencyMs - plan generation time in milliseconds

    • onPlanStepExecuted

      default void onPlanStepExecuted(EvalContext ctx, String actionName, boolean success, long latencyMs)
      Called when a single plan step is executed.
      Parameters:
      ctx - evaluation context
      actionName - the action that was executed
      success - whether the step succeeded
      latencyMs - step execution time in milliseconds

    • onPlanCompleted

      default void onPlanCompleted(EvalContext ctx, boolean success, int totalSteps, int executedSteps, long totalLatencyMs)
      Called when plan execution completes (all steps finished or a step failed).
      Parameters:
      ctx - evaluation context
      success - whether the entire plan succeeded
      totalSteps - total steps in the plan
      executedSteps - number of steps that executed successfully
      totalLatencyMs - total plan execution time in milliseconds

    • onPlanFailed

      default void onPlanFailed(EvalContext ctx, String goalId, String reason)
      Called when plan generation fails.
      Parameters:
      ctx - evaluation context
      goalId - the goal planning was attempted for
      reason - failure reason