TnsAI Evaluation 0.18.3 API
Packages
Package
Description
TnsAI Evaluation Framework - unified evaluation, benchmarking, and quality gating.
Self-improving agent evaluation loop (auto-harness pattern).
Layer 2: Benchmark engine for running test datasets, collecting results,
and detecting regressions against saved baselines.
Layer 1: Evaluators that score agent behavior on specific dimensions.
Agent-tier evaluators (TNS-373).
G-Eval: LLM-as-judge evaluation using chain-of-thought scoring.
Multi-turn conversation evaluation metrics.
RAG (Retrieval-Augmented Generation) evaluation metrics.
Safety evaluation metrics for LLM outputs.
Data model records for evaluation results, test cases, baselines, and configuration.
Opt-in multilingual embedding retrieval bench (TAN-2998).
Layer 3: Quality gates, evaluation reports, and trend analysis.
SPI interfaces for the evaluation framework.