Package com.tnsai.evaluation.evaluators.multiturn

Class TurnRelevancyEvaluator

java.lang.Object

com.tnsai.evaluation.evaluators.AbstractJudgeEvaluator

com.tnsai.evaluation.evaluators.multiturn.TurnRelevancyEvaluator

All Implemented Interfaces:

Evaluator

public final class TurnRelevancyEvaluator
extends AbstractJudgeEvaluator

Measures whether the last assistant turn is relevant to the preceding conversation context.

Uses LLM-as-judge to score on three dimensions:

• Context alignment: Does the response align with the conversation so far?
• Query addressing: Does it address the most recent user message?
• Coherence: Is it logically consistent with prior turns?

Final score is the average of the three normalized dimension scores.

Requires conversation_history in metadata as List<Map<String, String>> with "role" and "content".

Since:

0.3.0

Nested Class Summary

Nested classes/interfaces inherited from interface com.tnsai.evaluation.spi.Evaluator

Evaluator.EvaluationInput

Constructor Summary

Constructors

Constructor	Description
TurnRelevancyEvaluator(LLMJudge judge)

Method Summary

Methods inherited from class com.tnsai.evaluation.evaluators.AbstractJudgeEvaluator

evaluate, name

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

TurnRelevancyEvaluator

public TurnRelevancyEvaluator(LLMJudge judge)