Package com.tnsai.evaluation.evaluators.multiturn

Class KnowledgeRetentionEvaluator

java.lang.Object

com.tnsai.evaluation.evaluators.AbstractJudgeEvaluator

com.tnsai.evaluation.evaluators.multiturn.KnowledgeRetentionEvaluator

All Implemented Interfaces:

Evaluator

public final class KnowledgeRetentionEvaluator
extends AbstractJudgeEvaluator

Measures whether the agent retains information from earlier conversation turns.

Uses a 2-step LLM-as-judge process:

1. Extract key facts established in early turns
2. Check if the agent recalls those facts in later turns

Score = retained_facts / total_facts (1.0 = perfect retention, 0.0 = no retention).

Requires conversation_history in metadata as List<Map<String, String>> with "role" and "content".

Since:

0.3.0

Nested Class Summary

Nested classes/interfaces inherited from interface com.tnsai.evaluation.spi.Evaluator

Evaluator.EvaluationInput

Constructor Summary

Constructors

Constructor	Description
KnowledgeRetentionEvaluator(LLMJudge judge)

Method Summary

Methods inherited from class com.tnsai.evaluation.evaluators.AbstractJudgeEvaluator

evaluate, name

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

KnowledgeRetentionEvaluator

public KnowledgeRetentionEvaluator(LLMJudge judge)