Package com.tnsai.evaluation.evaluators.agent

Class PlanAdherenceEvaluator

java.lang.Object

com.tnsai.evaluation.evaluators.AbstractJudgeEvaluator

com.tnsai.evaluation.evaluators.agent.PlanAdherenceEvaluator

All Implemented Interfaces:

Evaluator

public final class PlanAdherenceEvaluator
extends AbstractJudgeEvaluator

Agent-tier metric: did the agent stick to its declared plan?

The "plan" here is whatever the caller wrote into Evaluator.EvaluationInput.instructions() — typically the system prompt, a TODO list the agent surfaced, or the fulfils/effects intent on a declared @ActionSpec. The judge LLM is asked to compare that declared plan against what the agent actually did (agentResponse + actualToolSequence) and produce a 1–5 rating that we normalise to [0.0, 1.0].

If the plan is absent or blank, the metric returns a failing result with a clear message rather than guessing — having no plan to compare against is a calling-site mistake, not a 1.0 "passes by default" outcome.

Companion to the deterministic StepEfficiencyEvaluator: adherence asks "did you do the right thing?", efficiency asks "did you do it without waste?". Both are agent-tier — they look at the trace, not just the final response text.

Since:

0.10.1 (TNS-373)

Nested Class Summary

Nested classes/interfaces inherited from interface com.tnsai.evaluation.spi.Evaluator

Evaluator.EvaluationInput

Constructor Summary

Constructors

Constructor	Description
PlanAdherenceEvaluator(LLMJudge judge)

Method Summary

Methods inherited from class com.tnsai.evaluation.evaluators.AbstractJudgeEvaluator

evaluate, name

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

PlanAdherenceEvaluator

public PlanAdherenceEvaluator(LLMJudge judge)