Package com.tnsai.llm.resilience

Class CircuitBreakerClient

java.lang.Object

com.tnsai.llm.resilience.CircuitBreakerClient

All Implemented Interfaces:

com.tnsai.llm.LLMClient

public class CircuitBreakerClient
extends Object
implements com.tnsai.llm.LLMClient

LLMClient decorator that implements the circuit breaker pattern.

Prevents cascading failures by fast-failing when a provider is consistently down, instead of wasting latency on every request.

States

• CLOSED — Normal operation. Failures are counted.
• OPEN — Fast-fail. All requests throw CircuitOpenException.
• HALF_OPEN — Probe mode. One request is allowed through to test recovery.

State Transitions

   CLOSED ──(N consecutive failures)──► OPEN
   OPEN ────(recovery timeout)────────► HALF_OPEN
   HALF_OPEN ──(probe succeeds)───────► CLOSED
   HALF_OPEN ──(probe fails)──────────► OPEN
 

Usage

 // Wrap any LLMClient
 LLMClient resilient = CircuitBreakerClient.wrap(openaiClient);

 // With custom settings
 LLMClient resilient = CircuitBreakerClient.builder()
     .client(openaiClient)
     .failureThreshold(3)
     .recoveryTimeout(Duration.ofSeconds(60))
     .build();

 // Compose with FallbackRouter for automatic failover
 FallbackRouter router = FallbackRouter.of(
     CircuitBreakerClient.wrap(primary),
     CircuitBreakerClient.wrap(fallback)
 );
 

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	CircuitBreakerClient.Builder
static final record	CircuitBreakerClient.CircuitBreakerMetrics	Snapshot of circuit breaker metrics.
static enum	CircuitBreakerClient.State	Circuit breaker states.

Method Summary

Modifier and Type	Method	Description
static CircuitBreakerClient.Builder	builder()
com.tnsai.llm.ChatResponse	chat(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
com.tnsai.llm.ChatResponse	chat(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools, float temperature)
com.tnsai.llm.ChatResponse	chat(List<com.tnsai.models.chat.ContentPart> messageParts, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
com.tnsai.llm.LLMCapabilities	getCapabilities()
int	getConsecutiveFailures()	Gets the number of consecutive failures.
com.tnsai.llm.LLMClient	getDelegate()	Gets the underlying delegate client.
Optional<Integer>	getMaxTokens()
CircuitBreakerClient.CircuitBreakerMetrics	getMetrics()	Gets circuit breaker metrics.
String	getModel()
CircuitBreakerClient.State	getState()	Gets the current circuit state.
float	getTemperature()
void	reset()	Manually resets the circuit breaker to CLOSED state.
Stream<String>	streamChat(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
Stream<String>	streamChat(List<com.tnsai.models.chat.ContentPart> messageParts, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
void	streamChatWithHandler(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools, Consumer<com.tnsai.llm.ChatChunk> handler)
Stream<com.tnsai.llm.ChatChunk>	streamChatWithSpec(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
static CircuitBreakerClient	wrap(com.tnsai.llm.LLMClient client)	Wraps a client with default circuit breaker settings (5 failures, 30s recovery).

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.tnsai.llm.LLMClient

chat, chat, chat, getProviderName, getTopP, listModels, streamChat, streamChatWithHandler, streamChatWithSpec

Method Details

builder

public static CircuitBreakerClient.Builder builder()

wrap

public static CircuitBreakerClient wrap(com.tnsai.llm.LLMClient client)

Wraps a client with default circuit breaker settings (5 failures, 30s recovery).

getModel

public String getModel()

Specified by:

getModel in interface com.tnsai.llm.LLMClient

getTemperature

public float getTemperature()

Specified by:

getTemperature in interface com.tnsai.llm.LLMClient

getMaxTokens

public Optional<Integer> getMaxTokens()

Specified by:

getMaxTokens in interface com.tnsai.llm.LLMClient

getCapabilities

public com.tnsai.llm.LLMCapabilities getCapabilities()

Specified by:

getCapabilities in interface com.tnsai.llm.LLMClient

chat

public com.tnsai.llm.ChatResponse chat(String message,
 Optional<String> systemPrompt,
 Optional<List<Map<String,Object>>> history,
 Optional<List<Map<String,Object>>> tools)

Specified by:

chat in interface com.tnsai.llm.LLMClient

chat

public com.tnsai.llm.ChatResponse chat(List<com.tnsai.models.chat.ContentPart> messageParts,
 Optional<String> systemPrompt,
 Optional<List<Map<String,Object>>> history,
 Optional<List<Map<String,Object>>> tools)

Specified by:

chat in interface com.tnsai.llm.LLMClient

chat

public com.tnsai.llm.ChatResponse chat(String message,
 Optional<String> systemPrompt,
 Optional<List<Map<String,Object>>> history,
 Optional<List<Map<String,Object>>> tools,
 float temperature)

Specified by:

chat in interface com.tnsai.llm.LLMClient

streamChat

public Stream<String> streamChat(String message,
 Optional<String> systemPrompt,
 Optional<List<Map<String,Object>>> history,
 Optional<List<Map<String,Object>>> tools)

Specified by:

streamChat in interface com.tnsai.llm.LLMClient

streamChat

public Stream<String> streamChat(List<com.tnsai.models.chat.ContentPart> messageParts,
 Optional<String> systemPrompt,
 Optional<List<Map<String,Object>>> history,
 Optional<List<Map<String,Object>>> tools)

Specified by:

streamChat in interface com.tnsai.llm.LLMClient

streamChatWithSpec

public Stream<com.tnsai.llm.ChatChunk> streamChatWithSpec(String message,
 Optional<String> systemPrompt,
 Optional<List<Map<String,Object>>> history,
 Optional<List<Map<String,Object>>> tools)

Specified by:

streamChatWithSpec in interface com.tnsai.llm.LLMClient

streamChatWithHandler

public void streamChatWithHandler(String message,
 Optional<String> systemPrompt,
 Optional<List<Map<String,Object>>> history,
 Optional<List<Map<String,Object>>> tools,
 Consumer<com.tnsai.llm.ChatChunk> handler)

Specified by:

streamChatWithHandler in interface com.tnsai.llm.LLMClient

getState

public CircuitBreakerClient.State getState()

Gets the current circuit state.

getConsecutiveFailures

public int getConsecutiveFailures()

Gets the number of consecutive failures.

getDelegate

public com.tnsai.llm.LLMClient getDelegate()

Gets the underlying delegate client.

getMetrics

public CircuitBreakerClient.CircuitBreakerMetrics getMetrics()

Gets circuit breaker metrics.

reset

public void reset()

Manually resets the circuit breaker to CLOSED state.