Package com.tnsai.memory.strategy

Class TieredContextLoader

java.lang.Object

com.tnsai.memory.strategy.TieredContextLoader

public final class TieredContextLoader
extends Object

Loads conversation context in tiers with token budgets.

Instead of loading all history and then pruning, TieredContextLoader assembles context from three tiers with independent token budgets:

1. Tier 1 (recent): Last N messages loaded verbatim -- highest priority
2. Tier 2 (summary): Older messages summarized by LLM -- medium priority
3. Tier 3 (search): Semantically relevant past messages -- lowest priority

Token Budget Allocation

 Total Budget (e.g., 4000 tokens)
 +-- Tier 1 (recent):  takes what it needs (up to totalBudget - summary - search)
 +-- Tier 2 (summary): up to summaryTokenBudget (e.g., 500)
 +-- Tier 3 (search):  up to searchTokenBudget (e.g., 300)
 

Usage Example

 TieredContextLoader loader = TieredContextLoader.builder()
     .store(memoryStore)
     .summarizer(messages -> llm.summarize(messages))
     .config(TieredContextLoader.TierConfig.defaults())
     .build();

 LoadedContext ctx = loader.load("What did we discuss about caching?");
 List<Map<String, Object>> messages = ctx.toMessageList();
 

Since:

0.1.0

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final class	TieredContextLoader.Builder	Builder for TieredContextLoader.
static interface	TieredContextLoader.LLMSummarizer	Functional interface for summarizing a list of messages.
static final record	TieredContextLoader.LoadedContext	Result of tiered context loading.
static final record	TieredContextLoader.TierConfig	Configuration for tier token budgets.

Method Summary

Modifier and Type	Method	Description
static TieredContextLoader.Builder	builder()	Creates a new builder.
TieredContextLoader.LoadedContext	load(String currentQuery)	Loads tiered context for the given query.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

builder

public static TieredContextLoader.Builder builder()

Creates a new builder.

load

public TieredContextLoader.LoadedContext load(String currentQuery)

Loads tiered context for the given query.

Assembles context by:

1. Taking the last recentTurnCount * 2 messages verbatim (user + assistant pairs)
2. Summarizing older messages (if any exist)
3. Searching for semantically relevant past messages matching the query

Parameters:

currentQuery - the current user query (used for semantic search), may be null

Returns:

loaded context with token estimates