Package com.tnsai.evaluation.autoharness

Class AutoHarnessLoop

java.lang.Object

com.tnsai.evaluation.autoharness.AutoHarnessLoop

public final class AutoHarnessLoop
extends Object

Orchestrates the self-improving evaluation loop: mine → cluster → generate → gate.

"Program the loop, not the agent." Each iteration mines new failures, clusters them by root cause, generates regression eval cases, and gates changes against the accumulated eval suite.

Usage

 AutoHarnessLoop loop = AutoHarnessLoop.builder()
     .miner(failureMiner)
     .clusterer(new SimpleFailureClusterer())
     .generator(new DefaultEvalCaseGenerator())
     .runner(benchmarkRunner)
     .passRateThreshold(0.8)
     .build();

 AutoHarnessLoop.LoopResult result = loop.run();
 if (result.gateResult().passed()) { // safe to proceed }
 

Since:

0.1.0

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final class	AutoHarnessLoop.Builder
static final record	AutoHarnessLoop.LoopResult	Result of a single auto-harness loop iteration.

Method Summary

Modifier and Type	Method	Description
static AutoHarnessLoop.Builder	builder()
AutoHarnessLoop.LoopResult	run()	Executes one full loop iteration: mine, cluster, generate, gate.
List<AutoHarnessLoop.LoopResult>	runUntilPass()	Runs the loop up to maxIterations times, accumulating results.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

run

public AutoHarnessLoop.LoopResult run()

Executes one full loop iteration: mine, cluster, generate, gate.

Returns:

result of the loop iteration

runUntilPass

public List<AutoHarnessLoop.LoopResult> runUntilPass()

Runs the loop up to maxIterations times, accumulating results. Stops early if the gate passes.

Returns:

list of results from each iteration

builder

public static AutoHarnessLoop.Builder builder()