Package com.tnsai.evaluation.evaluators.rag

Class AnswerRelevancyEvaluator

java.lang.Object

com.tnsai.evaluation.evaluators.AbstractJudgeEvaluator

com.tnsai.evaluation.evaluators.rag.AnswerRelevancyEvaluator

All Implemented Interfaces:

Evaluator

public final class AnswerRelevancyEvaluator
extends AbstractJudgeEvaluator

Measures whether the agent's response actually addresses the user's query.

Uses LLM-as-judge to score on three dimensions:

• Directness: Does the response directly answer the question?
• Completeness: Does it cover all aspects of the query?
• Focus: Does it avoid irrelevant tangents?

Final score is the average of the three normalized dimension scores.

Since:

0.3.0

Nested Class Summary

Nested classes/interfaces inherited from interface com.tnsai.evaluation.spi.Evaluator

Evaluator.EvaluationInput

Constructor Summary

Constructors

Constructor	Description
AnswerRelevancyEvaluator(LLMJudge judge)

Method Summary

Methods inherited from class com.tnsai.evaluation.evaluators.AbstractJudgeEvaluator

evaluate, name

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

AnswerRelevancyEvaluator

public AnswerRelevancyEvaluator(LLMJudge judge)