Package com.tnsai.llm.fallback

Interface LLMFallbackChain

All Superinterfaces:

com.tnsai.llm.LLMClient

public interface LLMFallbackChain
extends com.tnsai.llm.LLMClient

An LLMClient that wraps a primary plus a list of LLMFallbackRules, transparently routing failed calls to the first matching, not-cooling-down fallback target (issue #89).

Production deployments hit failure modes that one-shot retry with the same model can't recover from — provider outage, context-window overflow, persistent rate limit, model overload. A chain encodes graceful-degradation rules declaratively, so the agent layer above stays oblivious: it calls LLMClient.chat(java.lang.String, java.util.Optional<java.lang.String>, java.util.Optional<java.util.List<java.util.Map<java.lang.String, java.lang.Object>>>, java.util.Optional<java.util.List<java.util.Map<java.lang.String, java.lang.Object>>>) as usual and either gets an answer (possibly from a fallback) or, if every option is exhausted, an LLMChainExhaustedException that lists everything tried.

Example

 LLMFallbackChain chain = LLMFallbackChain.builder()
     .primary(openAIClient)
     // Context overflow → switch to a long-context model
     .fallback(claudeOpus1MClient,
               FallbackCondition.onErrorType(LLMException.ErrorType.CONTEXT_TOO_LONG))
     // Rate limit / overload on the primary → try Anthropic, hold off primary 60s
     .fallback(claudeSonnet4Client,
               FallbackCondition.onErrorType(
                   LLMException.ErrorType.MODEL_OVERLOADED,
                   LLMException.ErrorType.SERVER_ERROR),
               Duration.ofSeconds(60))
     // Anything else → cheap local catch-all
     .fallback(ollamaClient, FallbackCondition.always())
     .build();

 Agent agent = AgentBuilder.create().llm(chain).role(myRole).build();
 

Strict mode

Replay / evaluation runs need deterministic reproduction — fallbacks can quietly route the second run to a different model, masking a regression. Setting strictMode(true) disables every fallback; the chain becomes equivalent to its primary alone, throwing the primary's error directly. (Wiring into SessionReplayer from #81 is a follow-up — that issue isn't built yet.)

Since:

0.5.0 (issue #89)

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static final class	LLMFallbackChain.Builder	Fluent configurator.

Method Summary

Modifier and Type	Method	Description
static LLMFallbackChain.Builder	builder()	Returns a builder for configuring a chain.
static LLMFallbackChain	of(com.tnsai.llm.LLMClient primary, com.tnsai.llm.LLMClient... fallbacks)	Convenience: build a chain with default policy (try each in order on any LLMException, no cooldowns, no strict mode).
com.tnsai.llm.LLMClient	primary()	The chain's primary target — tried first on every call.
List<LLMFallbackRule>	rules()	Ordered list of fallback rules.
boolean	strictMode()	When true, fallbacks are disabled — the primary's failure propagates unchanged.

Methods inherited from interface com.tnsai.llm.LLMClient

chat, chat, chat, chat, chat, chat, getCapabilities, getMaxTokens, getModel, getProviderName, getTemperature, getTopP, listModels, streamChat, streamChat, streamChat, streamChatWithHandler, streamChatWithHandler, streamChatWithSpec, streamChatWithSpec

Method Details

primary

com.tnsai.llm.LLMClient primary()

The chain's primary target — tried first on every call.

rules

List<LLMFallbackRule> rules()

Ordered list of fallback rules. Evaluation walks this list and picks the first rule whose FallbackCondition matches the primary failure AND whose target is not currently cooling down.

strictMode

boolean strictMode()

When true, fallbacks are disabled — the primary's failure propagates unchanged. Used by replay / eval to guarantee the second run uses the same target as the first.

builder

static LLMFallbackChain.Builder builder()

Returns a builder for configuring a chain.

of

static LLMFallbackChain of(com.tnsai.llm.LLMClient primary,
 com.tnsai.llm.LLMClient... fallbacks)

Convenience: build a chain with default policy (try each in order on any LLMException, no cooldowns, no strict mode).