Package dev.agenor.core.memory.llm

Interface ContextWindowStrategy

All Known Implementing Classes:
FixedWindowStrategy, SlidingWindowStrategy, SummarizationStrategy
public interface ContextWindowStrategy
Strategy for selecting messages to include in LLM context window.

When conversation history exceeds the model's context window, a strategy is needed to decide which messages to include. Different strategies optimize for different goals:

  • Recency: Most recent messages (simple, fast)
  • Importance: Most important messages (requires scoring)
  • Summarization: Recent + summary of old (preserves context)
  • Semantic: Most relevant to current query (requires embeddings)

Built-in Implementations:

Concrete implementations are available in the runtime module:

  • Fixed window - Last N messages that fit
  • Sliding window - Recent + important messages
  • Summarization - Recent + summary of old

Implementations are typically accessed through a factory class provided by the runtime module.

Example Usage: 


 // Obtain strategy from runtime
 ContextWindowStrategy strategy = ...; // Provided by runtime

 // Use strategy to select messages
 List<LLMMessage> selected = strategy.selectMessages(
     allMessages,
     2000,
     tokenEstimator
 );

 // Use in LLM request
 LLMRequest request = LLMRequest.builder()
     .messages(selected)
     .maxTokens(500)
     .build();

Custom Strategies: 


 public class ImportanceStrategy implements ContextWindowStrategy {

     {@literal @}Override
     public List<LLMMessage> selectMessages(
         List<LLMMessage> allMessages,
         int maxTokens,
         TokenEstimator estimator
     ) {
         // Score messages by importance
         // Select highest scoring that fit in budget
         return selectedMessages;
     }

     {@literal @}Override
     public String getName() {
         return "importance";
     }
 }

Thread Safety: Implementations must be thread-safe.

Since:
0.6.0

  • Method Summary

    Modifier and Type
    Method
    Description
    String
    getName()
    Get strategy name for logging and debugging.
    default int
    getOverheadTokens()
    Get estimated overhead tokens for this strategy.
    default boolean
    requiresLLM()
    Check if this strategy requires an LLM provider.
    List<LLMMessage>
    selectMessages(List<LLMMessage> allMessages, int maxTokens, TokenEstimator estimator)
    Select messages to include in context window.

  • Method Details

    • selectMessages

      List<LLMMessage> selectMessages(List<LLMMessage> allMessages, int maxTokens, TokenEstimator estimator)
      Select messages to include in context window.

      Implementations must:

      • Return messages that fit within maxTokens budget
      • Preserve message order (oldest to newest)
      • Use provided TokenEstimator for token counting
      • Handle edge cases (empty list, budget too small, etc.)

      Example Implementation: 

      
 public List<LLMMessage> selectMessages(
     List<LLMMessage> allMessages,
     int maxTokens,
     TokenEstimator estimator
 ) {
     List<LLMMessage> selected = new ArrayList<>();
     int currentTokens = 0;

     // Start from end (most recent)
     for (int i = allMessages.size() - 1; i >= 0; i--) {
         LLMMessage msg = allMessages.get(i);
         int msgTokens = estimator.estimateTokens(msg);

         if (currentTokens + msgTokens <= maxTokens) {
             selected.add(0, msg);  // Add at start to maintain order
             currentTokens += msgTokens;
         } else {
             break;  // Budget exhausted
         }
     }

     return selected;
 }
      Parameters:
      allMessages - all available messages (oldest to newest)
      maxTokens - maximum tokens for selected messages
      estimator - token estimator to use
      Returns:
      selected messages that fit in budget (oldest to newest)
      Throws:
      IllegalArgumentException - if any parameter is null or maxTokens <= 0

    • getName

      String getName()
      Get strategy name for logging and debugging.

      Examples:

      • "fixed" - Fixed window strategy
      • "sliding" - Sliding window strategy
      • "summarized" - Summarization strategy
      • "semantic" - Semantic relevance strategy
      Returns:
      strategy name (lowercase, no spaces)

    • requiresLLM

      default boolean requiresLLM()
      Check if this strategy requires an LLM provider.

      Some strategies (like summarization) need an LLM to generate summaries. Others (like fixed window) do not.

      Returns:
      true if strategy needs LLM access

    • getOverheadTokens

      default int getOverheadTokens()
      Get estimated overhead tokens for this strategy.

      Some strategies add overhead:

      • Fixed/Sliding: 0 tokens (no modifications)
      • Summarized: ~100-300 tokens (summary message)
      • Semantic: 0 tokens (just selection)
      Returns:
      estimated overhead in tokens