Package com.tnsai.evaluation.evaluators.rag

Class FaithfulnessEvaluator

java.lang.Object

com.tnsai.evaluation.evaluators.AbstractJudgeEvaluator

com.tnsai.evaluation.evaluators.rag.FaithfulnessEvaluator

All Implemented Interfaces:

Evaluator

public final class FaithfulnessEvaluator
extends AbstractJudgeEvaluator

Measures whether the agent's response is grounded in retrieved documents.

Uses a 2-step LLM-as-judge process:

1. Extract factual claims from the response
2. Verify each claim against the retrieved context

Score = supported_claims / total_claims (1.0 = fully faithful, 0.0 = fully hallucinated).

Requires retrieved_documents in metadata as List<String>.

Since:

0.3.0

Nested Class Summary

Nested classes/interfaces inherited from interface com.tnsai.evaluation.spi.Evaluator

Evaluator.EvaluationInput

Constructor Summary

Constructors

Constructor	Description
FaithfulnessEvaluator(LLMJudge judge)

Method Summary

Methods inherited from class com.tnsai.evaluation.evaluators.AbstractJudgeEvaluator

evaluate, name

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

FaithfulnessEvaluator

public FaithfulnessEvaluator(LLMJudge judge)