Package com.tnsai.llm.cache
package com.tnsai.llm.cache
LLM response caching with semantic similarity and prompt caching support.
Provides embedding-based semantic caching via SemanticCache
and provider-level prompt caching via PromptCachingClient
to reduce API costs and latency.
- See Also:
-
ClassDescriptionWrapper for content that can be cached.A content block with optional cache control.Cache control configuration for prompt caching.Cache control type.LLM client wrapper that adds semantic caching capabilities.Builder for CachedLLMClient.Represents a cached LLM response entry.Cache usage statistics from an LLM response.Provides embeddings for text to enable semantic similarity search.The task an embedding is generated for.Simple hash-based embedding provider for testing and development.In-memory implementation of SemanticCache.Builder for InMemorySemanticCache.
EmbeddingProviderbacked by an Ollama server's/api/embedendpoint.LLMClient wrapper that adds prompt caching support.Semantic cache for LLM responses based on prompt similarity.Cache statistics.Result of a similarity search.