Package com.tnsai.evaluation.evaluators.agent

Class StepEfficiencyEvaluator

java.lang.Object
com.tnsai.evaluation.evaluators.agent.StepEfficiencyEvaluator
All Implemented Interfaces:
Evaluator
public final class StepEfficiencyEvaluator extends Object implements Evaluator
Agent-tier metric: did the agent reach the goal without burning extra tool calls?

Compares the actual tool-call count against an expected minimum. The expected minimum may come from either expectedToolSequence (when the caller has a known-correct trace) or from a numeric expected_steps entry in metadata (when the caller only knows the budget, not the exact tools).

Score is expected / max(expected, actual) — perfect efficiency when actual ≤ expected, then linearly degrades as the agent burns extra steps. Returns a passing score with a "no benchmark" note when neither expected source is populated.

This is the deterministic counterpart to the LLM-judged PlanAdherenceEvaluator / ToolCorrectnessEvaluator — cheap, repeatable, suitable for CI gates and dashboards.

Since:
0.10.1 (TNS-373)

  • Constructor Details

    • StepEfficiencyEvaluator

      public StepEfficiencyEvaluator()

  • Method Details

    • name

      public String name()
      Description copied from interface: Evaluator
      Unique name for this evaluator.
      Specified by:
      name in interface Evaluator

    • evaluate

      public EvaluationResult evaluate(Evaluator.EvaluationInput context)
      Description copied from interface: Evaluator
      Evaluates agent behavior for a single interaction.
      Specified by:
      evaluate in interface Evaluator
      Parameters:
      context - evaluation context containing input, output, tool calls, etc.
      Returns:
      evaluation result