Context Management

When conversations grow beyond a model's context window, Headroom decides which messages to keep and which to drop. Instead of naively removing the oldest messages, IntelligentContext scores every message by learned importance and drops the least valuable ones first.

IntelligentContext

IntelligentContext is a message-level compressor. It analyzes your conversation, assigns an importance score to each message, and removes low-scoring messages until the conversation fits within the token budget.

Dropped messages are not lost -- they are stored in CCR for on-demand retrieval by the LLM.

100-message conversation (50K tokens) with a 32K budget
  -> Score each message by importance
  -> Drop 60 lowest-scoring messages
  -> Cache dropped messages in CCR (hash=def456)
  -> Insert marker: "60 messages dropped, retrieve: def456"
  -> Final context: 40 messages within budget

Scoring weights

Each message receives a weighted score from six factors:

Weights are automatically normalized to sum to 1.0, so you can set relative values without worrying about exact proportions.

Rolling window fallback

If IntelligentContext is disabled or scoring data is unavailable, Headroom falls back to a rolling window strategy:

• Drop the oldest messages first
• Always keep the system prompt
• Always keep the last N user/assistant turns
• Drop tool calls and their responses as atomic pairs (no orphaned tool data)

This provides a safe baseline that works without any learned data.

Protection rules

Headroom enforces several protections to ensure model output quality:

Output buffer reservation

A configurable number of tokens is reserved for the model's response. The context budget is calculated as:

context_budget = model_context_limit - output_buffer_tokens

This prevents the input from consuming the entire context window and leaving no room for the model to respond.

System message protection

System messages are never dropped. They contain critical instructions, persona definitions, and tool descriptions that the model needs throughout the conversation.

Turn protection

The last N user/assistant turns are always preserved, ensuring the model has immediate conversational context. By default, the last 2 turns are protected.

Configuration

import { compress } from "headroom-ai";
import type {
  IntelligentContextConfig,
  ScoringWeights,
  RollingWindowConfig,
  HeadroomConfig,
} from "headroom-ai";

// Scoring weights (normalized automatically)
const scoringWeights: ScoringWeights = {
  recency: 0.20,
  semanticSimilarity: 0.20,
  toinImportance: 0.25,
  errorIndicator: 0.15,
  forwardReference: 0.15,
  tokenDensity: 0.05,
};

// IntelligentContext configuration
const intelligentContext: IntelligentContextConfig = {
  enabled: true,
  keepSystem: true,
  keepLastTurns: 2,
  outputBufferTokens: 4000,
  useImportanceScoring: true,
  scoringWeights,
  toinIntegration: true,
  recencyDecayRate: 0.1,
  compressThreshold: 0.1,
};

// Rolling window fallback
const rollingWindow: RollingWindowConfig = {
  enabled: true,
  keepSystem: true,
  keepLastTurns: 3,
  outputBufferTokens: 4000,
};

// Full configuration
const config: HeadroomConfig = {
  intelligentContext,
  rollingWindow,
};

const result = await compress(messages, {
  model: "gpt-4o",
  config,
});

console.log(`Compressed: ${result.tokensBefore} -> ${result.tokensAfter}`);

from headroom import HeadroomClient, OpenAIProvider
from headroom.config import IntelligentContextConfig, ScoringWeights
from openai import OpenAI

# Customize scoring weights
weights = ScoringWeights(
    recency=0.20,
    semantic_similarity=0.20,
    toin_importance=0.25,
    error_indicator=0.15,
    forward_reference=0.15,
    token_density=0.05,
)

context_config = IntelligentContextConfig(
    enabled=True,
    keep_system=True,            # Never drop system messages
    keep_last_turns=2,           # Protect last 2 user turns
    output_buffer_tokens=4000,   # Reserve for model output
    use_importance_scoring=True,
    scoring_weights=weights,
    toin_integration=True,       # Use TOIN patterns
    recency_decay_rate=0.1,      # Exponential decay lambda
    compress_threshold=0.1,      # Try compression first if <10% over budget
)

client = HeadroomClient(
    original_client=OpenAI(),
    provider=OpenAIProvider(),
    default_mode="optimize",
)

# Per-request overrides
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    headroom_output_buffer_tokens=8000,  # More room for long responses
    headroom_keep_turns=5,               # Protect last 5 turns
)

How scoring improves over time

IntelligentContext integrates with TOIN (Tool-Output Intelligence Network) to learn from real usage:

1. Messages are dropped based on current scores
2. Dropped messages are stored in CCR
3. If the LLM retrieves a dropped message, TOIN records that pattern
4. Future conversations score similar message patterns higher
5. Drop accuracy improves across all users, not just within one session

This feedback loop means the system gets smarter the more it is used. Error messages that users frequently need are automatically preserved, while verbose success messages that nobody retrieves are dropped more aggressively.