Package com.tnsai.llm.cache

Class CachedLLMClient.Builder

java.lang.Object

com.tnsai.llm.cache.CachedLLMClient.Builder

Enclosing class:

CachedLLMClient

public static class CachedLLMClient.Builder
extends Object

Builder for CachedLLMClient.

Method Summary

Modifier and Type	Method	Description
CachedLLMClient	build()
CachedLLMClient.Builder	cacheStreaming(boolean cache)	Whether to cache streaming responses.
CachedLLMClient.Builder	highThreshold(double threshold)	Sets the high similarity threshold for direct cache hits.
CachedLLMClient.Builder	lowThreshold(double threshold)	Sets the low similarity threshold.
CachedLLMClient.Builder	withCache(SemanticCache cache)	Sets the semantic cache to use.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

withCache

public CachedLLMClient.Builder withCache(SemanticCache cache)

Sets the semantic cache to use.

highThreshold

public CachedLLMClient.Builder highThreshold(double threshold)

Sets the high similarity threshold for direct cache hits. Default: 0.95

lowThreshold

public CachedLLMClient.Builder lowThreshold(double threshold)

Sets the low similarity threshold. Similarities below this skip the cache entirely. Default: 0.70

cacheStreaming

public CachedLLMClient.Builder cacheStreaming(boolean cache)

Whether to cache streaming responses. Default: true

build

public CachedLLMClient build()