Index
All Classes and Interfaces|All Packages|Constant Field Values
A
- absoluteChange() - Method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.MetricTrend
-
Returns the value of the
absoluteChangerecord component. - AbstractJudgeEvaluator - Class in com.tnsai.evaluation.evaluators
-
Base for single-call LLM-as-judge evaluators: build one prompt, send it to the
LLMJudgeonce, parse the verdict. - actualToolSequence() - Method in record class com.tnsai.evaluation.model.BenchmarkResult.TestCaseResult
-
Returns the value of the
actualToolSequencerecord component. - actualToolSequence() - Method in record class com.tnsai.evaluation.spi.Evaluator.EvaluationInput
-
Returns the value of the
actualToolSequencerecord component. - actualToolSequence(List<String>) - Method in class com.tnsai.evaluation.spi.Evaluator.EvaluationInput.Builder
- adapt(AgentTrace) - Method in class com.tnsai.evaluation.bridge.TraceToEvalAdapter
-
Converts an AgentTrace to an EvaluationInput.
- agentFunction(Function<TestCase, BenchmarkRunner.AgentOutput>) - Method in class com.tnsai.evaluation.benchmark.BenchmarkRunner.Builder
- agentId() - Method in record class com.tnsai.evaluation.autoharness.FailureRecord
-
Returns the value of the
agentIdrecord component. - agentId() - Method in record class com.tnsai.evaluation.bridge.TraceEvalBridge.FailedEvaluation
-
Returns the value of the
agentIdrecord component. - AgentOutput(String, List<String>, double, int, int) - Constructor for record class com.tnsai.evaluation.benchmark.BenchmarkRunner.AgentOutput
-
Creates an instance of a
AgentOutputrecord class. - agentResponse() - Method in record class com.tnsai.evaluation.model.BenchmarkResult.TestCaseResult
-
Returns the value of the
agentResponserecord component. - agentResponse() - Method in record class com.tnsai.evaluation.spi.Evaluator.EvaluationInput
-
Returns the value of the
agentResponserecord component. - agentResponse(String) - Method in class com.tnsai.evaluation.spi.Evaluator.EvaluationInput.Builder
- aggregateScores() - Method in record class com.tnsai.evaluation.model.BenchmarkResult
-
Returns the value of the
aggregateScoresrecord component. - analyze(List<Baseline>) - Static method in class com.tnsai.evaluation.reporting.TrendAnalyzer
-
Analyzes trends across ordered baselines (oldest first).
- analyze(List<Double>) - Static method in class com.tnsai.evaluation.benchmark.StatisticalAnalyzer
- annotate(AgentTrace, List<EvaluationResult>) - Method in class com.tnsai.evaluation.bridge.EvalSpanAnnotator
-
Annotates a trace with evaluation results as scores.
- AnswerRelevancyEvaluator - Class in com.tnsai.evaluation.evaluators.rag
-
Measures whether the agent's response actually addresses the user's query.
- AnswerRelevancyEvaluator(LLMJudge) - Constructor for class com.tnsai.evaluation.evaluators.rag.AnswerRelevancyEvaluator
- Architecture - Search tag in package com.tnsai.evaluation
- Section
- assertEval(String, Evaluator...) - Method in class com.tnsai.evaluation.junit.EvalAssert
-
Runs evaluators against the agent output and returns an assertion chain.
- assertEval(String, Map<String, Object>, Evaluator...) - Method in class com.tnsai.evaluation.junit.EvalAssert
-
Runs evaluators with additional metadata (tool sequences, latency, etc.).
- AutoHarnessLoop - Class in com.tnsai.evaluation.autoharness
-
Orchestrates the self-improving evaluation loop: mine → cluster → generate → gate.
- AutoHarnessLoop.Builder - Class in com.tnsai.evaluation.autoharness
- AutoHarnessLoop.LoopResult - Record Class in com.tnsai.evaluation.autoharness
-
Result of a single auto-harness loop iteration.
B
- Baseline - Record Class in com.tnsai.evaluation.model
-
A saved baseline from a benchmark run, used for regression detection.
- Baseline(String, String, Instant, Map<String, Double>, Map<String, Object>) - Constructor for record class com.tnsai.evaluation.model.Baseline
-
Creates an instance of a
Baselinerecord class. - BaselineStore - Class in com.tnsai.evaluation.benchmark
-
Persistent store for benchmark baselines.
- BaselineStore(Path) - Constructor for class com.tnsai.evaluation.benchmark.BaselineStore
- benchmarkResult() - Method in record class com.tnsai.evaluation.autoharness.RegressionGate.GateResult
-
Returns the value of the
benchmarkResultrecord component. - BenchmarkResult - Record Class in com.tnsai.evaluation.model
-
Aggregated result of running a benchmark dataset.
- BenchmarkResult(String, String, Instant, Instant, List<BenchmarkResult.TestCaseResult>, Map<String, Double>) - Constructor for record class com.tnsai.evaluation.model.BenchmarkResult
-
Creates an instance of a
BenchmarkResultrecord class. - BenchmarkResult.TestCaseResult - Record Class in com.tnsai.evaluation.model
-
Result for a single test case within a benchmark run.
- BenchmarkRunner - Class in com.tnsai.evaluation.benchmark
-
Runs benchmark datasets against an agent execution function, collecting evaluation results from registered evaluators.
- BenchmarkRunner.AgentOutput - Record Class in com.tnsai.evaluation.benchmark
-
Output from agent execution for a test case.
- BenchmarkRunner.Builder - Class in com.tnsai.evaluation.benchmark
- bestCases(int) - Method in class com.tnsai.evaluation.reporting.EvaluationReport
-
Returns the best-performing test cases.
- BiasEvaluator - Class in com.tnsai.evaluation.evaluators.safety
-
Detects demographic, cultural, or ideological bias in agent responses.
- BiasEvaluator(LLMJudge) - Constructor for class com.tnsai.evaluation.evaluators.safety.BiasEvaluator
- build() - Method in class com.tnsai.evaluation.autoharness.AutoHarnessLoop.Builder
- build() - Method in class com.tnsai.evaluation.benchmark.BenchmarkRunner.Builder
- build() - Method in class com.tnsai.evaluation.bridge.TraceEvalBridge.Builder
- build() - Method in class com.tnsai.evaluation.model.QualityGateConfig.Builder
- build() - Method in class com.tnsai.evaluation.spi.Evaluator.EvaluationInput.Builder
- builder() - Static method in class com.tnsai.evaluation.autoharness.AutoHarnessLoop
- builder() - Static method in class com.tnsai.evaluation.benchmark.BenchmarkRunner
- builder() - Static method in class com.tnsai.evaluation.bridge.TraceEvalBridge
- builder() - Static method in record class com.tnsai.evaluation.model.QualityGateConfig
-
Builder for fluent construction.
- builder() - Static method in record class com.tnsai.evaluation.spi.Evaluator.EvaluationInput
- Builder() - Constructor for class com.tnsai.evaluation.autoharness.AutoHarnessLoop.Builder
- Builder() - Constructor for class com.tnsai.evaluation.benchmark.BenchmarkRunner.Builder
- Builder() - Constructor for class com.tnsai.evaluation.bridge.TraceEvalBridge.Builder
- Builder() - Constructor for class com.tnsai.evaluation.model.QualityGateConfig.Builder
- Builder() - Constructor for class com.tnsai.evaluation.spi.Evaluator.EvaluationInput.Builder
- bySlice() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModelScore
- bytesPerDoc() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModel
-
Returns the value of the
bytesPerDocrecord component.
C
- caseResults() - Method in record class com.tnsai.evaluation.model.BenchmarkResult
-
Returns the value of the
caseResultsrecord component. - cases() - Method in record class com.tnsai.evaluation.model.TestDataset
-
Returns the value of the
casesrecord component. - cause() - Method in record class com.tnsai.evaluation.autoharness.FailureCluster
-
Returns the value of the
causerecord component. - changeProductionDefault() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchRecommendation.Decision
-
Returns the value of the
changeProductionDefaultrecord component. - ciLower() - Method in record class com.tnsai.evaluation.benchmark.StatisticalAnalyzer.Stats
-
Returns the value of the
ciLowerrecord component. - ciUpper() - Method in record class com.tnsai.evaluation.benchmark.StatisticalAnalyzer.Stats
-
Returns the value of the
ciUpperrecord component. - CLASSPATH_RESOURCE - Static variable in class com.tnsai.evaluation.rag.embedding.EmbeddingBenchCorpus
- clear() - Method in class com.tnsai.evaluation.autoharness.DefaultFailureMiner
- clear() - Method in interface com.tnsai.evaluation.autoharness.FailureMiner
-
Clears the internal failure buffer.
- cluster(List<FailureRecord>) - Method in interface com.tnsai.evaluation.autoharness.FailureClusterer
-
Clusters the given failures by root cause.
- cluster(List<FailureRecord>) - Method in class com.tnsai.evaluation.autoharness.SimpleFailureClusterer
- clusterCount() - Method in record class com.tnsai.evaluation.autoharness.AutoHarnessLoop.LoopResult
-
Returns the value of the
clusterCountrecord component. - clusterer(FailureClusterer) - Method in class com.tnsai.evaluation.autoharness.AutoHarnessLoop.Builder
- clusterKey() - Method in record class com.tnsai.evaluation.autoharness.FailureCluster
-
Returns the value of the
clusterKeyrecord component. - com.tnsai.evaluation - package com.tnsai.evaluation
-
TnsAI Evaluation Framework - unified evaluation, benchmarking, and quality gating.
- com.tnsai.evaluation.autoharness - package com.tnsai.evaluation.autoharness
-
Self-improving agent evaluation loop (auto-harness pattern).
- com.tnsai.evaluation.benchmark - package com.tnsai.evaluation.benchmark
-
Layer 2: Benchmark engine for running test datasets, collecting results, and detecting regressions against saved baselines.
- com.tnsai.evaluation.bridge - package com.tnsai.evaluation.bridge
- com.tnsai.evaluation.evaluators - package com.tnsai.evaluation.evaluators
-
Layer 1: Evaluators that score agent behavior on specific dimensions.
- com.tnsai.evaluation.evaluators.agent - package com.tnsai.evaluation.evaluators.agent
-
Agent-tier evaluators (TNS-373).
- com.tnsai.evaluation.evaluators.geval - package com.tnsai.evaluation.evaluators.geval
-
G-Eval: LLM-as-judge evaluation using chain-of-thought scoring.
- com.tnsai.evaluation.evaluators.multiturn - package com.tnsai.evaluation.evaluators.multiturn
-
Multi-turn conversation evaluation metrics.
- com.tnsai.evaluation.evaluators.rag - package com.tnsai.evaluation.evaluators.rag
-
RAG (Retrieval-Augmented Generation) evaluation metrics.
- com.tnsai.evaluation.evaluators.safety - package com.tnsai.evaluation.evaluators.safety
-
Safety evaluation metrics for LLM outputs.
- com.tnsai.evaluation.junit - package com.tnsai.evaluation.junit
- com.tnsai.evaluation.model - package com.tnsai.evaluation.model
-
Data model records for evaluation results, test cases, baselines, and configuration.
- com.tnsai.evaluation.rag.embedding - package com.tnsai.evaluation.rag.embedding
-
Opt-in multilingual embedding retrieval bench (TAN-2998).
- com.tnsai.evaluation.reporting - package com.tnsai.evaluation.reporting
-
Layer 3: Quality gates, evaluation reports, and trend analysis.
- com.tnsai.evaluation.spi - package com.tnsai.evaluation.spi
-
SPI interfaces for the evaluation framework.
- com.tnsai.evaluation.util - package com.tnsai.evaluation.util
- context() - Method in record class com.tnsai.evaluation.autoharness.FailureRecord
-
Returns the value of the
contextrecord component. - ContextualPrecisionEvaluator - Class in com.tnsai.evaluation.evaluators.rag
-
Measures whether retrieved documents are relevant to the query.
- ContextualPrecisionEvaluator(LLMJudge) - Constructor for class com.tnsai.evaluation.evaluators.rag.ContextualPrecisionEvaluator
- ContextualRecallEvaluator - Class in com.tnsai.evaluation.evaluators.rag
-
Measures whether all relevant information needed for the answer was retrieved.
- ContextualRecallEvaluator(LLMJudge) - Constructor for class com.tnsai.evaluation.evaluators.rag.ContextualRecallEvaluator
- ConversationCompletenessEvaluator - Class in com.tnsai.evaluation.evaluators.multiturn
-
Measures whether a multi-turn conversation achieved its stated goal.
- ConversationCompletenessEvaluator(LLMJudge) - Constructor for class com.tnsai.evaluation.evaluators.multiturn.ConversationCompletenessEvaluator
- Core abstractions - Search tag in package com.tnsai.evaluation.autoharness
- Section
- CostEfficiencyEvaluator - Class in com.tnsai.evaluation.evaluators
-
Evaluates cost efficiency: cost per successful task completion.
- CostEfficiencyEvaluator() - Constructor for class com.tnsai.evaluation.evaluators.CostEfficiencyEvaluator
-
Creates with default budgets: $0.10 cost, 30s latency.
- CostEfficiencyEvaluator(double, long) - Constructor for class com.tnsai.evaluation.evaluators.CostEfficiencyEvaluator
-
Creates with custom budgets.
- costUsd() - Method in record class com.tnsai.evaluation.benchmark.BenchmarkRunner.AgentOutput
-
Returns the value of the
costUsdrecord component. - costUsd() - Method in record class com.tnsai.evaluation.model.BenchmarkResult.TestCaseResult
-
Returns the value of the
costUsdrecord component. - costUsd() - Method in record class com.tnsai.evaluation.spi.Evaluator.EvaluationInput
-
Returns the value of the
costUsdrecord component. - costUsd(double) - Method in class com.tnsai.evaluation.spi.Evaluator.EvaluationInput.Builder
- count() - Method in record class com.tnsai.evaluation.autoharness.FailureCluster
-
Returns the value of the
countrecord component. - count() - Method in record class com.tnsai.evaluation.benchmark.StatisticalAnalyzer.Stats
-
Returns the value of the
countrecord component. - CREDIT_CARD - Enum constant in enum class com.tnsai.evaluation.evaluators.safety.PIILeakageEvaluator.Category
- CROSS - Enum constant in enum class com.tnsai.evaluation.rag.embedding.EmbeddingBenchSlice
D
- dataset() - Element in annotation interface com.tnsai.evaluation.junit.EvalTest
-
Path to the dataset JSON file.
- datasetName() - Method in record class com.tnsai.evaluation.model.Baseline
-
Returns the value of the
datasetNamerecord component. - datasetName() - Method in record class com.tnsai.evaluation.model.BenchmarkResult
-
Returns the value of the
datasetNamerecord component. - decide(List<EmbeddingBenchModelScore>) - Static method in class com.tnsai.evaluation.rag.embedding.EmbeddingBenchRecommendation
- Decision(String, boolean, String, String, String, String, String) - Constructor for record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchRecommendation.Decision
-
Creates an instance of a
Decisionrecord class. - DefaultEvalCaseGenerator - Class in com.tnsai.evaluation.autoharness
-
Generates BenchmarkRunner-compatible regression test cases from failure clusters.
- DefaultEvalCaseGenerator() - Constructor for class com.tnsai.evaluation.autoharness.DefaultEvalCaseGenerator
- DefaultEvalCaseGenerator(int) - Constructor for class com.tnsai.evaluation.autoharness.DefaultEvalCaseGenerator
- DefaultFailureMiner - Class in com.tnsai.evaluation.autoharness
-
Thread-safe
FailureMinerthat collects failures reported viaDefaultFailureMiner.record(FailureRecord). - DefaultFailureMiner() - Constructor for class com.tnsai.evaluation.autoharness.DefaultFailureMiner
- defaults() - Static method in record class com.tnsai.evaluation.model.QualityGateConfig
-
Default config: 0.7 threshold, 5% max regression, 80% pass rate.
- defaultThreshold() - Method in record class com.tnsai.evaluation.model.QualityGateConfig
-
Returns the value of the
defaultThresholdrecord component. - defaultThreshold(double) - Method in class com.tnsai.evaluation.model.QualityGateConfig.Builder
- DEGRADING - Enum constant in enum class com.tnsai.evaluation.reporting.TrendAnalyzer.Direction
- degradingMetrics() - Method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.TrendReport
- description() - Method in record class com.tnsai.evaluation.model.TestDataset
-
Returns the value of the
descriptionrecord component. - details() - Method in record class com.tnsai.evaluation.bridge.TraceEvalBridge.FailedEvaluation
-
Returns the value of the
detailsrecord component. - details() - Method in record class com.tnsai.evaluation.model.EvaluationResult
-
Returns the value of the
detailsrecord component. - DIM_384 - Static variable in class com.tnsai.evaluation.rag.embedding.TokenBagEmbeddingFunction
- dimensions() - Method in class com.tnsai.evaluation.rag.embedding.TokenBagEmbeddingFunction
- direction() - Method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.MetricTrend
-
Returns the value of the
directionrecord component. - document() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchPair
-
Returns the value of the
documentrecord component. - documentEmbedder() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModel
-
Returns the value of the
documentEmbedderrecord component. - documentId() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchPair
-
Returns the value of the
documentIdrecord component. - duration() - Method in record class com.tnsai.evaluation.model.BenchmarkResult
E
- EMAIL - Enum constant in enum class com.tnsai.evaluation.evaluators.safety.PIILeakageEvaluator.Category
- embed(String) - Method in class com.tnsai.evaluation.rag.embedding.OllamaEmbeddingFunction
- embed(String) - Method in class com.tnsai.evaluation.rag.embedding.TokenBagEmbeddingFunction
- EmbeddingBenchCorpus - Class in com.tnsai.evaluation.rag.embedding
-
Fixed TR/EN retrieval fixture shipped with the evaluation module.
- EmbeddingBenchModel - Record Class in com.tnsai.evaluation.rag.embedding
-
Named embedder under comparison.
- EmbeddingBenchModel(String, int, int, int, EmbeddingFunction, EmbeddingFunction) - Constructor for record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModel
-
Creates an instance of a
EmbeddingBenchModelrecord class. - EmbeddingBenchModelScore - Record Class in com.tnsai.evaluation.rag.embedding
-
Per-model scorecard across every slice.
- EmbeddingBenchModelScore(EmbeddingBenchModel, List<EmbeddingBenchSliceScore>) - Constructor for record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModelScore
-
Creates an instance of a
EmbeddingBenchModelScorerecord class. - EmbeddingBenchPair - Record Class in com.tnsai.evaluation.rag.embedding
-
One query and its gold document inside a
EmbeddingBenchSlice. - EmbeddingBenchPair(String, EmbeddingBenchSlice, String, String, String) - Constructor for record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchPair
-
Creates an instance of a
EmbeddingBenchPairrecord class. - EmbeddingBenchRecommendation - Class in com.tnsai.evaluation.rag.embedding
-
Picks a default candidate and a quality override from measured scores.
- EmbeddingBenchRecommendation.Decision - Record Class in com.tnsai.evaluation.rag.embedding
- EmbeddingBenchReport - Class in com.tnsai.evaluation.rag.embedding
-
Renders a markdown table for the measured model × slice matrix.
- EmbeddingBenchRunner - Class in com.tnsai.evaluation.rag.embedding
-
Runs the bilingual fixture against one or more
EmbeddingBenchModels. - EmbeddingBenchRunner(EmbeddingBenchCorpus) - Constructor for class com.tnsai.evaluation.rag.embedding.EmbeddingBenchRunner
- EmbeddingBenchSlice - Enum Class in com.tnsai.evaluation.rag.embedding
-
Language slice of the bilingual retrieval fixture.
- EmbeddingBenchSliceScore - Record Class in com.tnsai.evaluation.rag.embedding
-
Retrieval metrics for one language slice.
- EmbeddingBenchSliceScore(EmbeddingBenchSlice, int, double, double, double) - Constructor for record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchSliceScore
-
Creates an instance of a
EmbeddingBenchSliceScorerecord class. - embedLatencyP50Ms() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchSliceScore
-
Returns the value of the
embedLatencyP50Msrecord component. - embedQuery(String) - Method in class com.tnsai.evaluation.rag.embedding.OllamaEmbeddingFunction
-
Embeds a retrieval query with the model-specific query prefix.
- EMPTY - Static variable in record class com.tnsai.evaluation.benchmark.StatisticalAnalyzer.Stats
- EN - Enum constant in enum class com.tnsai.evaluation.rag.embedding.EmbeddingBenchSlice
- endTime() - Method in record class com.tnsai.evaluation.model.BenchmarkResult
-
Returns the value of the
endTimerecord component. - equals(Object) - Method in record class com.tnsai.evaluation.autoharness.AutoHarnessLoop.LoopResult
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.autoharness.EvalCase
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.autoharness.FailureCluster
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.autoharness.FailureRecord
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.autoharness.RegressionGate.GateResult
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.benchmark.BenchmarkRunner.AgentOutput
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.benchmark.StatisticalAnalyzer.Stats
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.bridge.TraceEvalBridge.FailedEvaluation
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.model.Baseline
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.model.BenchmarkResult
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.model.BenchmarkResult.TestCaseResult
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.model.EvaluationResult
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.model.QualityGateConfig
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.model.TestCase
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.model.TestDataset
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModel
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModelScore
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchPair
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchRecommendation.Decision
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchSliceScore
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.reporting.QualityGate.Verdict
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.MetricTrend
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.TrendReport
-
Indicates whether some other object is "equal to" this one.
- equals(Object) - Method in record class com.tnsai.evaluation.spi.Evaluator.EvaluationInput
-
Indicates whether some other object is "equal to" this one.
- error(String) - Static method in record class com.tnsai.evaluation.benchmark.BenchmarkRunner.AgentOutput
- errorType() - Method in record class com.tnsai.evaluation.autoharness.FailureRecord
-
Returns the value of the
errorTyperecord component. - EvalAssert - Class in com.tnsai.evaluation.junit
-
Fluent assertion API for evaluation tests.
- EvalAssert.EvalAssertion - Class in com.tnsai.evaluation.junit
-
Assertion chain for evaluation results.
- EvalCase - Record Class in com.tnsai.evaluation.autoharness
-
A generated regression test case derived from a
FailureCluster. - EvalCase(String, String, String, String, List<String>, Map<String, String>) - Constructor for record class com.tnsai.evaluation.autoharness.EvalCase
-
Creates an instance of a
EvalCaserecord class. - evalCaseCount() - Method in record class com.tnsai.evaluation.autoharness.AutoHarnessLoop.LoopResult
-
Returns the value of the
evalCaseCountrecord component. - EvalCaseGenerator - Interface in com.tnsai.evaluation.autoharness
-
Converts
FailureClusters into regressionEvalCases. - EvalExtension - Class in com.tnsai.evaluation.junit
-
JUnit 5 extension that powers
EvalTest. - EvalExtension() - Constructor for class com.tnsai.evaluation.junit.EvalExtension
- EvalSpanAnnotator - Class in com.tnsai.evaluation.bridge
-
Writes evaluation scores back onto
AgentTracespans for observability dashboards. - EvalSpanAnnotator() - Constructor for class com.tnsai.evaluation.bridge.EvalSpanAnnotator
- EvalTest - Annotation Interface in com.tnsai.evaluation.junit
-
Marks a method as an evaluation test.
- evaluate(BenchmarkResult) - Method in class com.tnsai.evaluation.reporting.QualityGate
-
Evaluates a benchmark result against the quality gate.
- evaluate(Evaluator.EvaluationInput) - Method in class com.tnsai.evaluation.evaluators.AbstractJudgeEvaluator
- evaluate(Evaluator.EvaluationInput) - Method in class com.tnsai.evaluation.evaluators.agent.StepEfficiencyEvaluator
- evaluate(Evaluator.EvaluationInput) - Method in class com.tnsai.evaluation.evaluators.CostEfficiencyEvaluator
- evaluate(Evaluator.EvaluationInput) - Method in class com.tnsai.evaluation.evaluators.geval.GEvalEvaluator
- evaluate(Evaluator.EvaluationInput) - Method in class com.tnsai.evaluation.evaluators.InstructionFollowingEvaluator
- evaluate(Evaluator.EvaluationInput) - Method in class com.tnsai.evaluation.evaluators.JsonCorrectnessEvaluator
- evaluate(Evaluator.EvaluationInput) - Method in class com.tnsai.evaluation.evaluators.rag.ContextualPrecisionEvaluator
- evaluate(Evaluator.EvaluationInput) - Method in class com.tnsai.evaluation.evaluators.ResponseQualityEvaluator
- evaluate(Evaluator.EvaluationInput) - Method in class com.tnsai.evaluation.evaluators.safety.PIILeakageEvaluator
- evaluate(Evaluator.EvaluationInput) - Method in class com.tnsai.evaluation.evaluators.ToolSelectionEvaluator
- evaluate(Evaluator.EvaluationInput) - Method in interface com.tnsai.evaluation.spi.Evaluator
-
Evaluates agent behavior for a single interaction.
- evaluate(List<EvalCase>) - Method in class com.tnsai.evaluation.autoharness.RegressionGate
-
Runs the eval cases and returns the gate result.
- EvaluationInput(String, String, String, List<String>, List<String>, String, long, double, int, int, Map<String, Object>) - Constructor for record class com.tnsai.evaluation.spi.Evaluator.EvaluationInput
-
Creates an instance of a
EvaluationInputrecord class. - EvaluationReport - Class in com.tnsai.evaluation.reporting
-
Aggregated evaluation report with drill-down capability.
- EvaluationReport(BenchmarkResult, QualityGate.Verdict) - Constructor for class com.tnsai.evaluation.reporting.EvaluationReport
- EvaluationResult - Record Class in com.tnsai.evaluation.model
-
Result of a single evaluation.
- EvaluationResult(String, double, String, Map<String, Double>, Instant) - Constructor for record class com.tnsai.evaluation.model.EvaluationResult
-
Creates an instance of a
EvaluationResultrecord class. - evaluationResults() - Method in record class com.tnsai.evaluation.model.BenchmarkResult.TestCaseResult
-
Returns the value of the
evaluationResultsrecord component. - evaluator(Evaluator) - Method in class com.tnsai.evaluation.benchmark.BenchmarkRunner.Builder
- evaluator(Evaluator) - Method in class com.tnsai.evaluation.bridge.TraceEvalBridge.Builder
- Evaluator - Interface in com.tnsai.evaluation.spi
-
SPI interface for all evaluators.
- Evaluator.EvaluationInput - Record Class in com.tnsai.evaluation.spi
-
Input context provided to evaluators.
- Evaluator.EvaluationInput.Builder - Class in com.tnsai.evaluation.spi
- evaluatorName() - Method in record class com.tnsai.evaluation.bridge.TraceEvalBridge.FailedEvaluation
-
Returns the value of the
evaluatorNamerecord component. - evaluatorName() - Method in record class com.tnsai.evaluation.model.EvaluationResult
-
Returns the value of the
evaluatorNamerecord component. - evaluators(List<Evaluator>) - Method in class com.tnsai.evaluation.benchmark.BenchmarkRunner.Builder
- evaluators(List<Evaluator>) - Method in class com.tnsai.evaluation.bridge.TraceEvalBridge.Builder
- evaluatorStats() - Method in class com.tnsai.evaluation.reporting.EvaluationReport
-
Returns per-evaluator statistical summary.
- evaluatorThreshold(String, double) - Method in class com.tnsai.evaluation.model.QualityGateConfig.Builder
- evaluatorThresholds() - Method in record class com.tnsai.evaluation.model.QualityGateConfig
-
Returns the value of the
evaluatorThresholdsrecord component. - examples() - Method in record class com.tnsai.evaluation.autoharness.FailureCluster
-
Returns the value of the
examplesrecord component. - expectedBehavior() - Method in record class com.tnsai.evaluation.autoharness.EvalCase
-
Returns the value of the
expectedBehaviorrecord component. - expectedOutput() - Method in record class com.tnsai.evaluation.model.TestCase
-
Returns the value of the
expectedOutputrecord component. - expectedOutput() - Method in record class com.tnsai.evaluation.spi.Evaluator.EvaluationInput
-
Returns the value of the
expectedOutputrecord component. - expectedOutput(String) - Method in class com.tnsai.evaluation.spi.Evaluator.EvaluationInput.Builder
- expectedToolSequence() - Method in record class com.tnsai.evaluation.model.TestCase
-
Returns the value of the
expectedToolSequencerecord component. - expectedToolSequence() - Method in record class com.tnsai.evaluation.spi.Evaluator.EvaluationInput
-
Returns the value of the
expectedToolSequencerecord component. - expectedToolSequence(List<String>) - Method in class com.tnsai.evaluation.spi.Evaluator.EvaluationInput.Builder
- exportJson(Path) - Method in class com.tnsai.evaluation.reporting.EvaluationReport
-
Exports the report to a JSON file.
F
- fail(String, String) - Static method in record class com.tnsai.evaluation.model.EvaluationResult
-
Creates a simple fail result.
- FailedEvaluation(String, String, String, double, String, Evaluator.EvaluationInput) - Constructor for record class com.tnsai.evaluation.bridge.TraceEvalBridge.FailedEvaluation
-
Creates an instance of a
FailedEvaluationrecord class. - fails() - Method in class com.tnsai.evaluation.junit.EvalAssert.EvalAssertion
-
Assert all evaluators scored below the threshold (useful for toxicity/bias checks).
- FailureCluster - Record Class in com.tnsai.evaluation.autoharness
-
A group of
FailureRecords sharing a common root cause. - FailureCluster(String, String, int, List<FailureRecord>) - Constructor for record class com.tnsai.evaluation.autoharness.FailureCluster
-
Creates an instance of a
FailureClusterrecord class. - FailureClusterer - Interface in com.tnsai.evaluation.autoharness
-
Groups
FailureRecords intoFailureClusters by root cause. - failureCount() - Method in record class com.tnsai.evaluation.autoharness.AutoHarnessLoop.LoopResult
-
Returns the value of the
failureCountrecord component. - FailureMiner - Interface in com.tnsai.evaluation.autoharness
-
Extracts
FailureRecords from agent execution traces. - FailureRecord - Record Class in com.tnsai.evaluation.autoharness
-
Captures a single agent failure from an execution trace.
- FailureRecord(String, String, String, String, Instant, Map<String, String>) - Constructor for record class com.tnsai.evaluation.autoharness.FailureRecord
-
Creates an instance of a
FailureRecordrecord class. - failures() - Method in record class com.tnsai.evaluation.reporting.QualityGate.Verdict
-
Returns the value of the
failuresrecord component. - failureThreshold(double) - Method in class com.tnsai.evaluation.bridge.TraceEvalBridge.Builder
- FaithfulnessEvaluator - Class in com.tnsai.evaluation.evaluators.rag
-
Measures whether the agent's response is grounded in retrieved documents.
- FaithfulnessEvaluator(LLMJudge) - Constructor for class com.tnsai.evaluation.evaluators.rag.FaithfulnessEvaluator
- filterByTag(String) - Method in record class com.tnsai.evaluation.model.TestDataset
-
Filters test cases by tag.
- firstValue() - Method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.MetricTrend
-
Returns the value of the
firstValuerecord component. - fromBenchmarkResult(BenchmarkResult) - Static method in record class com.tnsai.evaluation.model.Baseline
- fromClasspath(String) - Static method in record class com.tnsai.evaluation.model.TestDataset
-
Loads a dataset from classpath.
- fromFile(Path) - Static method in record class com.tnsai.evaluation.model.TestDataset
-
Loads a dataset from a JSON file.
G
- gatePassed() - Method in record class com.tnsai.evaluation.autoharness.AutoHarnessLoop.LoopResult
- gateResult() - Method in record class com.tnsai.evaluation.autoharness.AutoHarnessLoop.LoopResult
-
Returns the value of the
gateResultrecord component. - GateResult(boolean, double, int, int, BenchmarkResult, String) - Constructor for record class com.tnsai.evaluation.autoharness.RegressionGate.GateResult
-
Creates an instance of a
GateResultrecord class. - generate(List<FailureCluster>) - Method in class com.tnsai.evaluation.autoharness.DefaultEvalCaseGenerator
- generate(List<FailureCluster>) - Method in interface com.tnsai.evaluation.autoharness.EvalCaseGenerator
-
Generates eval cases from the given failure clusters.
- generator(EvalCaseGenerator) - Method in class com.tnsai.evaluation.autoharness.AutoHarnessLoop.Builder
- getBenchmarkResult() - Method in class com.tnsai.evaluation.reporting.EvaluationReport
- getVerdict() - Method in class com.tnsai.evaluation.reporting.EvaluationReport
- GEvalEvaluator - Class in com.tnsai.evaluation.evaluators.geval
-
G-Eval: LLM-as-judge evaluator using a 2-step chain-of-thought process.
- GEvalEvaluator(String, String, LLMJudge) - Constructor for class com.tnsai.evaluation.evaluators.geval.GEvalEvaluator
-
Creates a G-Eval evaluator for a specific evaluation criteria.
H
- HallucinationEvaluator - Class in com.tnsai.evaluation.evaluators.safety
-
Detects hallucinated content in agent responses by checking factual claims against provided context.
- HallucinationEvaluator(LLMJudge) - Constructor for class com.tnsai.evaluation.evaluators.safety.HallucinationEvaluator
- hasDegradations() - Method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.TrendReport
- hashCode() - Method in record class com.tnsai.evaluation.autoharness.AutoHarnessLoop.LoopResult
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.autoharness.EvalCase
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.autoharness.FailureCluster
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.autoharness.FailureRecord
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.autoharness.RegressionGate.GateResult
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.benchmark.BenchmarkRunner.AgentOutput
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.benchmark.StatisticalAnalyzer.Stats
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.bridge.TraceEvalBridge.FailedEvaluation
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.model.Baseline
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.model.BenchmarkResult
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.model.BenchmarkResult.TestCaseResult
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.model.EvaluationResult
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.model.QualityGateConfig
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.model.TestCase
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.model.TestDataset
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModel
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModelScore
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchPair
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchRecommendation.Decision
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchSliceScore
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.reporting.QualityGate.Verdict
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.MetricTrend
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.TrendReport
-
Returns a hash code value for this object.
- hashCode() - Method in record class com.tnsai.evaluation.spi.Evaluator.EvaluationInput
-
Returns a hash code value for this object.
- How they relate - Search tag in package com.tnsai.evaluation.evaluators.agent
- Section
I
- IBAN - Enum constant in enum class com.tnsai.evaluation.evaluators.safety.PIILeakageEvaluator.Category
- id() - Method in record class com.tnsai.evaluation.autoharness.EvalCase
-
Returns the value of the
idrecord component. - id() - Method in record class com.tnsai.evaluation.model.TestCase
-
Returns the value of the
idrecord component. - id() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchPair
-
Returns the value of the
idrecord component. - IMPROVING - Enum constant in enum class com.tnsai.evaluation.reporting.TrendAnalyzer.Direction
- improvingMetrics() - Method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.TrendReport
- input() - Method in record class com.tnsai.evaluation.autoharness.EvalCase
-
Returns the value of the
inputrecord component. - input() - Method in record class com.tnsai.evaluation.bridge.TraceEvalBridge.FailedEvaluation
-
Returns the value of the
inputrecord component. - input() - Method in record class com.tnsai.evaluation.model.TestCase
-
Returns the value of the
inputrecord component. - inputTokens() - Method in record class com.tnsai.evaluation.benchmark.BenchmarkRunner.AgentOutput
-
Returns the value of the
inputTokensrecord component. - inputTokens() - Method in record class com.tnsai.evaluation.spi.Evaluator.EvaluationInput
-
Returns the value of the
inputTokensrecord component. - inputTokens(int) - Method in class com.tnsai.evaluation.spi.Evaluator.EvaluationInput.Builder
- installableDefault() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchRecommendation.Decision
-
Returns the value of the
installableDefaultrecord component. - InstructionFollowingEvaluator - Class in com.tnsai.evaluation.evaluators
-
Evaluates whether the agent followed given instructions.
- InstructionFollowingEvaluator() - Constructor for class com.tnsai.evaluation.evaluators.InstructionFollowingEvaluator
- instructions() - Method in record class com.tnsai.evaluation.model.TestCase
-
Returns the value of the
instructionsrecord component. - instructions() - Method in record class com.tnsai.evaluation.spi.Evaluator.EvaluationInput
-
Returns the value of the
instructionsrecord component. - instructions(String) - Method in class com.tnsai.evaluation.spi.Evaluator.EvaluationInput.Builder
- insufficient() - Static method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.TrendReport
- IP_ADDRESS - Enum constant in enum class com.tnsai.evaluation.evaluators.safety.PIILeakageEvaluator.Category
J
- JsonCorrectnessEvaluator - Class in com.tnsai.evaluation.evaluators
-
Evaluates JSON correctness of agent output — parse validity, required fields, and type conformance.
- JsonCorrectnessEvaluator() - Constructor for class com.tnsai.evaluation.evaluators.JsonCorrectnessEvaluator
- judge(String) - Method in interface com.tnsai.evaluation.evaluators.geval.LLMJudge
-
Sends a prompt to an LLM and returns the text response.
K
- KnowledgeRetentionEvaluator - Class in com.tnsai.evaluation.evaluators.multiturn
-
Measures whether the agent retains information from earlier conversation turns.
- KnowledgeRetentionEvaluator(LLMJudge) - Constructor for class com.tnsai.evaluation.evaluators.multiturn.KnowledgeRetentionEvaluator
L
- lastValue() - Method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.MetricTrend
-
Returns the value of the
lastValuerecord component. - latencyMs() - Method in record class com.tnsai.evaluation.model.BenchmarkResult.TestCaseResult
-
Returns the value of the
latencyMsrecord component. - latencyMs() - Method in record class com.tnsai.evaluation.spi.Evaluator.EvaluationInput
-
Returns the value of the
latencyMsrecord component. - latencyMs(long) - Method in class com.tnsai.evaluation.spi.Evaluator.EvaluationInput.Builder
- lcsRatio(List<String>, List<String>) - Static method in class com.tnsai.evaluation.util.SequenceUtils
-
Computes the Longest Common Subsequence ratio between two sequences.
- listBaselines(String) - Method in class com.tnsai.evaluation.benchmark.BaselineStore
-
Lists all baselines for a dataset.
- LLMJudge - Interface in com.tnsai.evaluation.evaluators.geval
-
Functional interface for LLM-based judging.
- load(String, String) - Method in class com.tnsai.evaluation.benchmark.BaselineStore
-
Loads a specific baseline.
- loadDefault() - Static method in class com.tnsai.evaluation.rag.embedding.EmbeddingBenchCorpus
-
Loads the bundled bilingual fixture from the evaluation test/main classpath.
- loadFrom(Path) - Static method in record class com.tnsai.evaluation.model.Baseline
- loadLatest(String) - Method in class com.tnsai.evaluation.benchmark.BaselineStore
-
Loads the latest baseline for a dataset.
- LoopResult(int, int, int, RegressionGate.GateResult, String) - Constructor for record class com.tnsai.evaluation.autoharness.AutoHarnessLoop.LoopResult
-
Creates an instance of a
LoopResultrecord class. - loser() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchRecommendation.Decision
-
Returns the value of the
loserrecord component. - loserStillUsefulWhen() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchRecommendation.Decision
-
Returns the value of the
loserStillUsefulWhenrecord component.
M
- markdown(List<EmbeddingBenchModelScore>, EmbeddingBenchRecommendation.Decision) - Static method in class com.tnsai.evaluation.rag.embedding.EmbeddingBenchReport
- max() - Method in record class com.tnsai.evaluation.benchmark.StatisticalAnalyzer.Stats
-
Returns the value of the
maxrecord component. - maxIterations(int) - Method in class com.tnsai.evaluation.autoharness.AutoHarnessLoop.Builder
-
Maximum number of loop iterations before giving up.
- maxRegressionPercent() - Method in record class com.tnsai.evaluation.model.QualityGateConfig
-
Returns the value of the
maxRegressionPercentrecord component. - maxRegressionPercent(double) - Method in class com.tnsai.evaluation.model.QualityGateConfig.Builder
- mean() - Method in record class com.tnsai.evaluation.benchmark.StatisticalAnalyzer.Stats
-
Returns the value of the
meanrecord component. - median() - Method in record class com.tnsai.evaluation.benchmark.StatisticalAnalyzer.Stats
-
Returns the value of the
medianrecord component. - message() - Method in record class com.tnsai.evaluation.autoharness.FailureRecord
-
Returns the value of the
messagerecord component. - metadata() - Method in record class com.tnsai.evaluation.autoharness.EvalCase
-
Returns the value of the
metadatarecord component. - metadata() - Method in record class com.tnsai.evaluation.model.Baseline
-
Returns the value of the
metadatarecord component. - metadata() - Method in record class com.tnsai.evaluation.model.TestCase
-
Returns the value of the
metadatarecord component. - metadata() - Method in record class com.tnsai.evaluation.spi.Evaluator.EvaluationInput
-
Returns the value of the
metadatarecord component. - metadata(String, Object) - Method in class com.tnsai.evaluation.spi.Evaluator.EvaluationInput.Builder
- metrics() - Method in record class com.tnsai.evaluation.model.EvaluationResult
-
Returns the value of the
metricsrecord component. - MetricTrend(double, double, double, double, double, TrendAnalyzer.Direction) - Constructor for record class com.tnsai.evaluation.reporting.TrendAnalyzer.MetricTrend
-
Creates an instance of a
MetricTrendrecord class. - metricTrends() - Method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.TrendReport
-
Returns the value of the
metricTrendsrecord component. - min() - Method in record class com.tnsai.evaluation.benchmark.StatisticalAnalyzer.Stats
-
Returns the value of the
minrecord component. - mine() - Method in class com.tnsai.evaluation.autoharness.DefaultFailureMiner
- mine() - Method in interface com.tnsai.evaluation.autoharness.FailureMiner
-
Mines failures accumulated since the last call (or since construction).
- miner(FailureMiner) - Method in class com.tnsai.evaluation.autoharness.AutoHarnessLoop.Builder
- minFailuresForClustering(int) - Method in class com.tnsai.evaluation.autoharness.AutoHarnessLoop.Builder
-
Minimum number of failures required before clustering is attempted.
- minPassRate() - Method in record class com.tnsai.evaluation.model.QualityGateConfig
-
Returns the value of the
minPassRaterecord component. - minPassRate(double) - Method in class com.tnsai.evaluation.model.QualityGateConfig.Builder
- model() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModelScore
-
Returns the value of the
modelrecord component. - model() - Method in class com.tnsai.evaluation.rag.embedding.OllamaEmbeddingFunction
- modelAvailable(String, String) - Static method in class com.tnsai.evaluation.rag.embedding.OllamaEmbeddingFunction
-
True when
/api/tagslistsmodel. - mrr() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchSliceScore
-
Returns the value of the
mrrrecord component.
N
- name() - Method in class com.tnsai.evaluation.evaluators.AbstractJudgeEvaluator
- name() - Method in class com.tnsai.evaluation.evaluators.agent.StepEfficiencyEvaluator
- name() - Method in class com.tnsai.evaluation.evaluators.CostEfficiencyEvaluator
- name() - Method in class com.tnsai.evaluation.evaluators.geval.GEvalEvaluator
- name() - Method in class com.tnsai.evaluation.evaluators.InstructionFollowingEvaluator
- name() - Method in class com.tnsai.evaluation.evaluators.JsonCorrectnessEvaluator
- name() - Method in class com.tnsai.evaluation.evaluators.rag.ContextualPrecisionEvaluator
- name() - Method in class com.tnsai.evaluation.evaluators.ResponseQualityEvaluator
- name() - Method in class com.tnsai.evaluation.evaluators.safety.PIILeakageEvaluator
- name() - Method in class com.tnsai.evaluation.evaluators.ToolSelectionEvaluator
- name() - Method in record class com.tnsai.evaluation.model.TestDataset
-
Returns the value of the
namerecord component. - name() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModel
-
Returns the value of the
namerecord component. - name() - Method in interface com.tnsai.evaluation.spi.Evaluator
-
Unique name for this evaluator.
- nativeDimensions() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModel
-
Returns the value of the
nativeDimensionsrecord component.
O
- of(String, double, String, Map<String, Double>) - Static method in record class com.tnsai.evaluation.model.EvaluationResult
-
Creates a result with current timestamp.
- of(String, int, int, EmbeddingFunction, EmbeddingFunction) - Static method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModel
- of(String, int, EmbeddingFunction) - Static method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModel
- of(String, String, String, String) - Static method in record class com.tnsai.evaluation.autoharness.FailureRecord
-
Convenience factory with explicit error info.
- of(String, String, Throwable) - Static method in record class com.tnsai.evaluation.autoharness.FailureRecord
-
Convenience factory for the common case.
- of(String, List<String>, double, int, int) - Static method in record class com.tnsai.evaluation.benchmark.BenchmarkRunner.AgentOutput
- ollama(String, int, OllamaEmbeddingFunction) - Static method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModel
-
Wraps an Ollama model so queries use the query prefix and passages use the document prefix.
- OllamaEmbeddingFunction - Class in com.tnsai.evaluation.rag.embedding
-
EmbeddingFunctionover an Ollama/api/embedendpoint. - OllamaEmbeddingFunction(String, String, String, String) - Constructor for class com.tnsai.evaluation.rag.embedding.OllamaEmbeddingFunction
- onFailure(Consumer<TraceEvalBridge.FailedEvaluation>) - Method in class com.tnsai.evaluation.bridge.TraceEvalBridge.Builder
- outputTokens() - Method in record class com.tnsai.evaluation.benchmark.BenchmarkRunner.AgentOutput
-
Returns the value of the
outputTokensrecord component. - outputTokens() - Method in record class com.tnsai.evaluation.spi.Evaluator.EvaluationInput
-
Returns the value of the
outputTokensrecord component. - outputTokens(int) - Method in class com.tnsai.evaluation.spi.Evaluator.EvaluationInput.Builder
- overallScore() - Method in record class com.tnsai.evaluation.model.BenchmarkResult.TestCaseResult
P
- p5() - Method in record class com.tnsai.evaluation.benchmark.StatisticalAnalyzer.Stats
-
Returns the value of the
p5record component. - p95() - Method in record class com.tnsai.evaluation.benchmark.StatisticalAnalyzer.Stats
-
Returns the value of the
p95record component. - pairs() - Method in class com.tnsai.evaluation.rag.embedding.EmbeddingBenchCorpus
- pairs(EmbeddingBenchSlice) - Method in class com.tnsai.evaluation.rag.embedding.EmbeddingBenchCorpus
- parallelism(int) - Method in class com.tnsai.evaluation.benchmark.BenchmarkRunner.Builder
- pass(String, String) - Static method in record class com.tnsai.evaluation.model.EvaluationResult
-
Creates a simple pass/fail result.
- passed() - Method in record class com.tnsai.evaluation.autoharness.RegressionGate.GateResult
-
Returns the value of the
passedrecord component. - passed() - Method in record class com.tnsai.evaluation.reporting.QualityGate.Verdict
-
Returns the value of the
passedrecord component. - passed(double) - Method in record class com.tnsai.evaluation.model.EvaluationResult
- passedCases(double) - Method in record class com.tnsai.evaluation.model.BenchmarkResult
- passedCount() - Method in record class com.tnsai.evaluation.autoharness.RegressionGate.GateResult
-
Returns the value of the
passedCountrecord component. - passes() - Method in class com.tnsai.evaluation.junit.EvalAssert.EvalAssertion
-
Assert all evaluators scored above the threshold.
- passRate() - Method in record class com.tnsai.evaluation.autoharness.RegressionGate.GateResult
-
Returns the value of the
passRaterecord component. - passRate() - Method in record class com.tnsai.evaluation.reporting.QualityGate.Verdict
-
Returns the value of the
passRaterecord component. - passRate(double) - Method in record class com.tnsai.evaluation.model.BenchmarkResult
- passRateThreshold(double) - Method in class com.tnsai.evaluation.autoharness.AutoHarnessLoop.Builder
-
Minimum fraction of eval cases that must pass [0.0, 1.0].
- percentChange() - Method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.MetricTrend
-
Returns the value of the
percentChangerecord component. - percentile(List<Double>, int) - Static method in class com.tnsai.evaluation.benchmark.StatisticalAnalyzer
-
Computes a specific percentile from a sorted list.
- phase() - Method in record class com.tnsai.evaluation.autoharness.FailureRecord
-
Returns the value of the
phaserecord component. - PHONE - Enum constant in enum class com.tnsai.evaluation.evaluators.safety.PIILeakageEvaluator.Category
- PIILeakageEvaluator - Class in com.tnsai.evaluation.evaluators.safety
-
Detects PII leakage in agent responses using regex + checksum validation.
- PIILeakageEvaluator() - Constructor for class com.tnsai.evaluation.evaluators.safety.PIILeakageEvaluator
- PIILeakageEvaluator(Set<PIILeakageEvaluator.Category>) - Constructor for class com.tnsai.evaluation.evaluators.safety.PIILeakageEvaluator
- PIILeakageEvaluator.Category - Enum Class in com.tnsai.evaluation.evaluators.safety
- PlanAdherenceEvaluator - Class in com.tnsai.evaluation.evaluators.agent
-
Agent-tier metric: did the agent stick to its declared plan?
- PlanAdherenceEvaluator(LLMJudge) - Constructor for class com.tnsai.evaluation.evaluators.agent.PlanAdherenceEvaluator
- Plug-in style - Search tag in package com.tnsai.evaluation.evaluators.agent
- Section
- process(AgentTrace) - Method in class com.tnsai.evaluation.bridge.TraceEvalBridge
-
Process a completed trace: adapt → evaluate → annotate → report failures.
- PRODUCTION_DEFAULT - Static variable in class com.tnsai.evaluation.rag.embedding.EmbeddingBenchRecommendation
- productionDefault() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchRecommendation.Decision
-
Returns the value of the
productionDefaultrecord component. - provideTestTemplateInvocationContexts(ExtensionContext) - Method in class com.tnsai.evaluation.junit.EvalExtension
Q
- QualityGate - Class in com.tnsai.evaluation.reporting
-
Pass/fail gate based on configurable thresholds and baseline regression detection.
- QualityGate(QualityGateConfig) - Constructor for class com.tnsai.evaluation.reporting.QualityGate
- QualityGate(QualityGateConfig, BaselineStore) - Constructor for class com.tnsai.evaluation.reporting.QualityGate
- QualityGate.Verdict - Record Class in com.tnsai.evaluation.reporting
-
Quality gate verdict.
- QualityGateConfig - Record Class in com.tnsai.evaluation.model
-
Configuration for quality gate pass/fail thresholds.
- QualityGateConfig(double, Map<String, Double>, double, double) - Constructor for record class com.tnsai.evaluation.model.QualityGateConfig
-
Creates an instance of a
QualityGateConfigrecord class. - QualityGateConfig.Builder - Class in com.tnsai.evaluation.model
- qualityOverride() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchRecommendation.Decision
-
Returns the value of the
qualityOverriderecord component. - queries() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchSliceScore
-
Returns the value of the
queriesrecord component. - query() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchPair
-
Returns the value of the
queryrecord component. - queryEmbedder() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModel
-
Returns the value of the
queryEmbedderrecord component.
R
- rationale() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchRecommendation.Decision
-
Returns the value of the
rationalerecord component. - recallAt5() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchSliceScore
-
Returns the value of the
recallAt5record component. - record(FailureRecord) - Method in class com.tnsai.evaluation.autoharness.DefaultFailureMiner
-
Records a failure into the mining buffer.
- record(String, String, String, String, Map<String, String>) - Method in class com.tnsai.evaluation.autoharness.DefaultFailureMiner
-
Records a failure from raw error information with context.
- record(String, String, Throwable) - Method in class com.tnsai.evaluation.autoharness.DefaultFailureMiner
-
Records a failure from raw error information.
- RegressionGate - Class in com.tnsai.evaluation.autoharness
-
Runs an eval suite and gates on a pass rate threshold.
- RegressionGate(BenchmarkRunner, double, double) - Constructor for class com.tnsai.evaluation.autoharness.RegressionGate
- RegressionGate.GateResult - Record Class in com.tnsai.evaluation.autoharness
-
Result of a regression gate evaluation.
- reportedDimensions() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModel
-
Returns the value of the
reportedDimensionsrecord component. - resolveParameter(ParameterContext, ExtensionContext) - Method in class com.tnsai.evaluation.junit.EvalExtension
- response() - Method in record class com.tnsai.evaluation.benchmark.BenchmarkRunner.AgentOutput
-
Returns the value of the
responserecord component. - ResponseQualityEvaluator - Class in com.tnsai.evaluation.evaluators
-
Evaluates response quality across three dimensions: relevance, completeness, and accuracy.
- ResponseQualityEvaluator() - Constructor for class com.tnsai.evaluation.evaluators.ResponseQualityEvaluator
- results() - Method in class com.tnsai.evaluation.junit.EvalAssert.EvalAssertion
-
Get the raw results for custom assertions.
- run() - Method in class com.tnsai.evaluation.autoharness.AutoHarnessLoop
-
Executes one full loop iteration: mine, cluster, generate, gate.
- run(TestDataset) - Method in class com.tnsai.evaluation.benchmark.BenchmarkRunner
-
Runs all test cases in the dataset and collects results.
- run(List<EmbeddingBenchModel>) - Method in class com.tnsai.evaluation.rag.embedding.EmbeddingBenchRunner
- runCount() - Method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.TrendReport
-
Returns the value of the
runCountrecord component. - runId() - Method in record class com.tnsai.evaluation.model.Baseline
-
Returns the value of the
runIdrecord component. - runId() - Method in record class com.tnsai.evaluation.model.BenchmarkResult
-
Returns the value of the
runIdrecord component. - runner(BenchmarkRunner) - Method in class com.tnsai.evaluation.autoharness.AutoHarnessLoop.Builder
- runUntilPass() - Method in class com.tnsai.evaluation.autoharness.AutoHarnessLoop
-
Runs the loop up to
maxIterationstimes, accumulating results.
S
- save(BenchmarkResult) - Method in class com.tnsai.evaluation.benchmark.BaselineStore
-
Saves a baseline from a benchmark result.
- saveTo(Path) - Method in record class com.tnsai.evaluation.model.Baseline
- score() - Method in record class com.tnsai.evaluation.bridge.TraceEvalBridge.FailedEvaluation
-
Returns the value of the
scorerecord component. - score() - Method in record class com.tnsai.evaluation.model.EvaluationResult
-
Returns the value of the
scorerecord component. - score(EmbeddingBenchSlice) - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModelScore
- scores() - Method in record class com.tnsai.evaluation.model.Baseline
-
Returns the value of the
scoresrecord component. - scores() - Method in record class com.tnsai.evaluation.reporting.QualityGate.Verdict
-
Returns the value of the
scoresrecord component. - scoresByEvaluator() - Method in record class com.tnsai.evaluation.model.BenchmarkResult.TestCaseResult
- scoreThreshold(double) - Method in class com.tnsai.evaluation.autoharness.AutoHarnessLoop.Builder
-
Minimum score for an individual case to be considered passing [0.0, 1.0].
- Scoring - Search tag in class com.tnsai.evaluation.evaluators.JsonCorrectnessEvaluator
- Section
- SequenceUtils - Class in com.tnsai.evaluation.util
-
Shared sequence comparison utilities for evaluators.
- simple(String) - Static method in record class com.tnsai.evaluation.benchmark.BenchmarkRunner.AgentOutput
- SimpleFailureClusterer - Class in com.tnsai.evaluation.autoharness
-
Groups
FailureRecords byerrorType + phase. - SimpleFailureClusterer() - Constructor for class com.tnsai.evaluation.autoharness.SimpleFailureClusterer
- SimpleFailureClusterer(int) - Constructor for class com.tnsai.evaluation.autoharness.SimpleFailureClusterer
- size() - Method in class com.tnsai.evaluation.autoharness.DefaultFailureMiner
-
Returns the current buffer size without copying.
- size() - Method in record class com.tnsai.evaluation.model.TestDataset
-
Returns number of test cases.
- size() - Method in class com.tnsai.evaluation.rag.embedding.EmbeddingBenchCorpus
- slice() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchPair
-
Returns the value of the
slicerecord component. - slice() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchSliceScore
-
Returns the value of the
slicerecord component. - slices() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModelScore
-
Returns the value of the
slicesrecord component. - slope() - Method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.MetricTrend
-
Returns the value of the
sloperecord component. - sourceClusterKey() - Method in record class com.tnsai.evaluation.autoharness.EvalCase
-
Returns the value of the
sourceClusterKeyrecord component. - STABLE - Enum constant in enum class com.tnsai.evaluation.reporting.TrendAnalyzer.Direction
- startTime() - Method in record class com.tnsai.evaluation.model.BenchmarkResult
-
Returns the value of the
startTimerecord component. - StatisticalAnalyzer - Class in com.tnsai.evaluation.benchmark
-
Statistical analysis utilities for evaluation scores.
- StatisticalAnalyzer.Stats - Record Class in com.tnsai.evaluation.benchmark
-
Statistical summary.
- Stats(int, double, double, double, double, double, double, double, double, double) - Constructor for record class com.tnsai.evaluation.benchmark.StatisticalAnalyzer.Stats
-
Creates an instance of a
Statsrecord class. - stdDev() - Method in record class com.tnsai.evaluation.benchmark.StatisticalAnalyzer.Stats
-
Returns the value of the
stdDevrecord component. - StepEfficiencyEvaluator - Class in com.tnsai.evaluation.evaluators.agent
-
Agent-tier metric: did the agent reach the goal without burning extra tool calls?
- StepEfficiencyEvaluator() - Constructor for class com.tnsai.evaluation.evaluators.agent.StepEfficiencyEvaluator
- summary() - Method in record class com.tnsai.evaluation.autoharness.AutoHarnessLoop.LoopResult
-
Returns the value of the
summaryrecord component. - summary() - Method in record class com.tnsai.evaluation.autoharness.RegressionGate.GateResult
-
Returns the value of the
summaryrecord component. - summary() - Method in record class com.tnsai.evaluation.reporting.QualityGate.Verdict
- supportsParameter(ParameterContext, ExtensionContext) - Method in class com.tnsai.evaluation.junit.EvalExtension
- supportsTestTemplate(ExtensionContext) - Method in class com.tnsai.evaluation.junit.EvalExtension
T
- tag() - Element in annotation interface com.tnsai.evaluation.junit.EvalTest
-
Filter test cases by tag.
- tags() - Method in record class com.tnsai.evaluation.autoharness.EvalCase
-
Returns the value of the
tagsrecord component. - tags() - Method in record class com.tnsai.evaluation.model.TestCase
-
Returns the value of the
tagsrecord component. - TaskCompletionEvaluator - Class in com.tnsai.evaluation.evaluators.agent
-
Agent-tier metric: did the agent's final state actually solve the task?
- TaskCompletionEvaluator(LLMJudge) - Constructor for class com.tnsai.evaluation.evaluators.agent.TaskCompletionEvaluator
- TCKN - Enum constant in enum class com.tnsai.evaluation.evaluators.safety.PIILeakageEvaluator.Category
- TestCase - Record Class in com.tnsai.evaluation.model
-
A single test case in a benchmark dataset.
- TestCase(String, String, String, List<String>, String, List<String>, Map<String, Object>) - Constructor for record class com.tnsai.evaluation.model.TestCase
-
Creates an instance of a
TestCaserecord class. - testCaseId() - Method in record class com.tnsai.evaluation.model.BenchmarkResult.TestCaseResult
-
Returns the value of the
testCaseIdrecord component. - TestCaseResult(String, String, List<String>, List<EvaluationResult>, long, double) - Constructor for record class com.tnsai.evaluation.model.BenchmarkResult.TestCaseResult
-
Creates an instance of a
TestCaseResultrecord class. - TestDataset - Record Class in com.tnsai.evaluation.model
-
A collection of test cases for benchmarking.
- TestDataset(String, String, String, List<TestCase>) - Constructor for record class com.tnsai.evaluation.model.TestDataset
-
Creates an instance of a
TestDatasetrecord class. - The four metrics - Search tag in package com.tnsai.evaluation.evaluators.agent
- Section
- threshold() - Element in annotation interface com.tnsai.evaluation.junit.EvalTest
-
Default score threshold for all evaluators (0.0-1.0).
- thresholdFor(String) - Method in record class com.tnsai.evaluation.model.QualityGateConfig
- timestamp() - Method in record class com.tnsai.evaluation.autoharness.FailureRecord
-
Returns the value of the
timestamprecord component. - timestamp() - Method in record class com.tnsai.evaluation.model.Baseline
-
Returns the value of the
timestamprecord component. - timestamp() - Method in record class com.tnsai.evaluation.model.EvaluationResult
-
Returns the value of the
timestamprecord component. - TokenBagEmbeddingFunction - Class in com.tnsai.evaluation.rag.embedding
-
Deterministic bag-of-tokens embedding used as an offline bench baseline.
- TokenBagEmbeddingFunction(int) - Constructor for class com.tnsai.evaluation.rag.embedding.TokenBagEmbeddingFunction
- toMap() - Method in record class com.tnsai.evaluation.benchmark.StatisticalAnalyzer.Stats
- toMap() - Method in class com.tnsai.evaluation.reporting.EvaluationReport
-
Generates a serializable map of the full report.
- ToolCorrectnessEvaluator - Class in com.tnsai.evaluation.evaluators.agent
-
Agent-tier metric: were the tools the agent invoked semantically appropriate for the task?
- ToolCorrectnessEvaluator(LLMJudge) - Constructor for class com.tnsai.evaluation.evaluators.agent.ToolCorrectnessEvaluator
- ToolSelectionEvaluator - Class in com.tnsai.evaluation.evaluators
-
Evaluates whether the agent selected the correct tools in the right order.
- ToolSelectionEvaluator() - Constructor for class com.tnsai.evaluation.evaluators.ToolSelectionEvaluator
- toolSequence() - Method in record class com.tnsai.evaluation.benchmark.BenchmarkRunner.AgentOutput
-
Returns the value of the
toolSequencerecord component. - TOP_K - Static variable in class com.tnsai.evaluation.rag.embedding.EmbeddingBenchRunner
- toString() - Method in record class com.tnsai.evaluation.autoharness.AutoHarnessLoop.LoopResult
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.autoharness.EvalCase
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.autoharness.FailureCluster
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.autoharness.FailureRecord
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.autoharness.RegressionGate.GateResult
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.benchmark.BenchmarkRunner.AgentOutput
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.benchmark.StatisticalAnalyzer.Stats
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.bridge.TraceEvalBridge.FailedEvaluation
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.model.Baseline
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.model.BenchmarkResult.TestCaseResult
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.model.BenchmarkResult
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.model.EvaluationResult
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.model.QualityGateConfig
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.model.TestCase
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.model.TestDataset
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModel
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchModelScore
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchPair
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchRecommendation.Decision
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.rag.embedding.EmbeddingBenchSliceScore
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.reporting.QualityGate.Verdict
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.MetricTrend
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.reporting.TrendAnalyzer.TrendReport
-
Returns a string representation of this record class.
- toString() - Method in record class com.tnsai.evaluation.spi.Evaluator.EvaluationInput
-
Returns a string representation of this record class.
- totalCases() - Method in record class com.tnsai.evaluation.model.BenchmarkResult
- totalCount() - Method in record class com.tnsai.evaluation.autoharness.RegressionGate.GateResult
-
Returns the value of the
totalCountrecord component. - toTestCase() - Method in record class com.tnsai.evaluation.autoharness.EvalCase
-
Converts this eval case into a BenchmarkRunner-compatible TestCase.
- ToxicityEvaluator - Class in com.tnsai.evaluation.evaluators.safety
-
Detects toxic, harmful, or offensive content in agent responses.
- ToxicityEvaluator(LLMJudge) - Constructor for class com.tnsai.evaluation.evaluators.safety.ToxicityEvaluator
- TR - Enum constant in enum class com.tnsai.evaluation.rag.embedding.EmbeddingBenchSlice
- TraceEvalBridge - Class in com.tnsai.evaluation.bridge
-
Bridges AgentTrace → Evaluator → score annotation pipeline.
- TraceEvalBridge.Builder - Class in com.tnsai.evaluation.bridge
- TraceEvalBridge.FailedEvaluation - Record Class in com.tnsai.evaluation.bridge
-
Record of a failed evaluation for downstream processing (e.g., AutoHarness).
- traceId() - Method in record class com.tnsai.evaluation.bridge.TraceEvalBridge.FailedEvaluation
-
Returns the value of the
traceIdrecord component. - TraceToEvalAdapter - Class in com.tnsai.evaluation.bridge
-
Converts completed
AgentTracespans intoEvaluator.EvaluationInputrecords. - TraceToEvalAdapter() - Constructor for class com.tnsai.evaluation.bridge.TraceToEvalAdapter
- TrendAnalyzer - Class in com.tnsai.evaluation.reporting
-
Compares evaluation results across multiple runs to detect trends.
- TrendAnalyzer.Direction - Enum Class in com.tnsai.evaluation.reporting
- TrendAnalyzer.MetricTrend - Record Class in com.tnsai.evaluation.reporting
- TrendAnalyzer.TrendReport - Record Class in com.tnsai.evaluation.reporting
- TrendReport(int, Map<String, TrendAnalyzer.MetricTrend>) - Constructor for record class com.tnsai.evaluation.reporting.TrendAnalyzer.TrendReport
-
Creates an instance of a
TrendReportrecord class. - TurnRelevancyEvaluator - Class in com.tnsai.evaluation.evaluators.multiturn
-
Measures whether the last assistant turn is relevant to the preceding conversation context.
- TurnRelevancyEvaluator(LLMJudge) - Constructor for class com.tnsai.evaluation.evaluators.multiturn.TurnRelevancyEvaluator
U
- Usage - Search tag in annotation interface com.tnsai.evaluation.junit.EvalTest
- Section
- Usage - Search tag in class com.tnsai.evaluation.autoharness.AutoHarnessLoop
- Section
- Usage - Search tag in class com.tnsai.evaluation.benchmark.BenchmarkRunner
- Section
- Usage - Search tag in class com.tnsai.evaluation.bridge.TraceEvalBridge
- Section
- Usage - Search tag in class com.tnsai.evaluation.evaluators.geval.GEvalEvaluator
- Section
- Usage - Search tag in class com.tnsai.evaluation.junit.EvalAssert
- Section
- userInput() - Method in record class com.tnsai.evaluation.spi.Evaluator.EvaluationInput
-
Returns the value of the
userInputrecord component. - userInput(String) - Method in class com.tnsai.evaluation.spi.Evaluator.EvaluationInput.Builder
V
- valueOf(String) - Static method in enum class com.tnsai.evaluation.evaluators.safety.PIILeakageEvaluator.Category
-
Returns the enum constant of this class with the specified name.
- valueOf(String) - Static method in enum class com.tnsai.evaluation.rag.embedding.EmbeddingBenchSlice
-
Returns the enum constant of this class with the specified name.
- valueOf(String) - Static method in enum class com.tnsai.evaluation.reporting.TrendAnalyzer.Direction
-
Returns the enum constant of this class with the specified name.
- values() - Static method in enum class com.tnsai.evaluation.evaluators.safety.PIILeakageEvaluator.Category
-
Returns an array containing the constants of this enum class, in the order they are declared.
- values() - Static method in enum class com.tnsai.evaluation.rag.embedding.EmbeddingBenchSlice
-
Returns an array containing the constants of this enum class, in the order they are declared.
- values() - Static method in enum class com.tnsai.evaluation.reporting.TrendAnalyzer.Direction
-
Returns an array containing the constants of this enum class, in the order they are declared.
- Verdict(boolean, List<String>, List<String>, double, Map<String, Double>) - Constructor for record class com.tnsai.evaluation.reporting.QualityGate.Verdict
-
Creates an instance of a
Verdictrecord class. - version() - Method in record class com.tnsai.evaluation.model.TestDataset
-
Returns the value of the
versionrecord component.
W
- warnings() - Method in record class com.tnsai.evaluation.reporting.QualityGate.Verdict
-
Returns the value of the
warningsrecord component. - withThreshold(double) - Method in class com.tnsai.evaluation.junit.EvalAssert.EvalAssertion
-
Override the default threshold for this assertion.
- worstCases(int) - Method in class com.tnsai.evaluation.reporting.EvaluationReport
-
Returns the worst-performing test cases.
All Classes and Interfaces|All Packages|Constant Field Values