Package com.tnsai.llm.cache

Class CacheControl

java.lang.Object
com.tnsai.llm.cache.CacheControl
public class CacheControl extends Object
Cache control configuration for prompt caching.

CacheControl is used to mark content for caching in LLM requests. Currently supports Anthropic's ephemeral caching strategy.

Anthropic Prompt Caching

Anthropic allows caching of:

  • System prompts
  • Tool definitions
  • Conversation history (up to last 4 cache breakpoints)

Cost Benefits

  • Cache writes: 25% more expensive than base input tokens
  • Cache reads: 90% cheaper than base input tokens
  • TTL: 5 minutes (refreshed on each use)

Usage Example


 // Mark system prompt for caching
 CacheablePrompt cachedSystem = CacheablePrompt.of(
     "You are an expert assistant...",
     CacheControl.ephemeral()
 );

 // Use in chat
 client.chat(message, cachedSystem);
See Also:

  • Method Details

    • ephemeral

      public static CacheControl ephemeral()
      Creates an ephemeral cache control.

      Ephemeral caches have a 5-minute TTL that is refreshed each time the cached content is used.

      Returns:
      Ephemeral cache control

    • getType

      public CacheControl.Type getType()
      Gets the cache control type.

    • getTypeValue

      public String getTypeValue()
      Gets the type value as string.

    • toString

      public String toString()
      Overrides:
      toString in class Object