Package com.tnsai.context.compaction

Class LLMContextCompactor

java.lang.Object
com.tnsai.context.compaction.LLMContextCompactor
All Implemented Interfaces:
ContextCompactor
public class LLMContextCompactor extends Object implements ContextCompactor
LLM-based context compactor that summarizes old messages.

Uses an LLM to generate a summary of older messages while preserving the most recent messages intact.

Usage


 LLMClient summarizer = LLMClientFactory.create("openai", "gpt-4o-mini");
 LLMContextCompactor compactor = new LLMContextCompactor(summarizer);

 CompactionConfig config = CompactionConfig.builder()
     .thresholdRatio(0.80)
     .preserveLastN(5)
     .build();

 CompactionResult result = compactor.compact(messages, config);

Compaction Process

  1. Separate system prompt (if preserveSystemPrompt=true)
  2. Identify messages to preserve (last N)
  3. Summarize remaining messages using LLM
  4. Reconstruct context: [system] + [summary] + [preserved]
Author:
TnsAI Team

  • Constructor Details

    • LLMContextCompactor

      public LLMContextCompactor(com.tnsai.llm.LLMClient llm)
      Creates a compactor with the given LLM client.
      Parameters:
      llm - the LLM client for summarization

  • Method Details