Package com.tnsai.evaluation.autoharness

Class RegressionGate

java.lang.Object

com.tnsai.evaluation.autoharness.RegressionGate

public final class RegressionGate
extends Object

Runs an eval suite and gates on a pass rate threshold.

Given a list of EvalCases, converts them into a TestDataset and executes via BenchmarkRunner. The gate passes if the overall pass rate meets the configured threshold.

Since:

0.1.0

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final record	RegressionGate.GateResult	Result of a regression gate evaluation.

Constructor Summary

Constructors

Constructor	Description
RegressionGate(BenchmarkRunner runner, double passRateThreshold, double scoreThreshold)

Method Summary

Modifier and Type	Method	Description
RegressionGate.GateResult	evaluate(List<EvalCase> evalCases)	Runs the eval cases and returns the gate result.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

RegressionGate

public RegressionGate(BenchmarkRunner runner,
 double passRateThreshold,
 double scoreThreshold)

Parameters:

runner - configured BenchmarkRunner to execute eval cases

passRateThreshold - fraction of cases that must pass [0.0, 1.0]

scoreThreshold - minimum score for a case to be considered passing [0.0, 1.0]

Method Details

evaluate

public RegressionGate.GateResult evaluate(List<EvalCase> evalCases)

Runs the eval cases and returns the gate result.

Parameters:

evalCases - generated regression test cases

Returns:

gate result with pass/fail and details