Context Management

Context management is now handled automatically inside the pipeline (live-zone-only compression). Headroom never drops messages from the conversation history and does not do position-based or score-based context management.

How It Works

Headroom compresses only the newest content blocks — the latest user message and the latest tool result / tool output. Compression is type-aware and reversible via CCR, so the LLM can retrieve the original content on demand.

The cache hot zone — the system prompt, tool definitions, and older turns — is never mutated. Leaving the prefix untouched preserves provider prompt caching, so cache hit rates stay stable across turns.

Conversation with a large latest tool result
  -> Identify the live zone (newest user message + latest tool output)
  -> Compress the live zone type-aware, cache original in CCR (hash=def456)
  -> Insert marker: "compressed, retrieve: def456"
  -> Older turns, tools, and system prompt are forwarded byte-for-byte

Protection rules

Headroom enforces several protections to ensure model output quality:

Output buffer reservation

A configurable number of tokens is reserved for the model's response. The context budget is calculated as:

context_budget = model_context_limit - output_buffer_tokens

This prevents the input from consuming the entire context window and leaving no room for the model to respond.

System message protection

System messages are never dropped. They contain critical instructions, persona definitions, and tool descriptions that the model needs throughout the conversation.

Turn protection

The last N user/assistant turns are always preserved, ensuring the model has immediate conversational context. By default, the last 2 turns are protected.

Configuration

Context management is now automatic. Use per-request overrides to control behavior:

import { compress } from "headroom-ai";

const result = await compress(messages, {
  model: "gpt-4o",
  tokenBudget: 32000,
});

console.log(`Compressed: ${result.tokensBefore} -> ${result.tokensAfter}`);

from headroom import HeadroomClient, OpenAIProvider
from openai import OpenAI

client = HeadroomClient(
    original_client=OpenAI(),
    provider=OpenAIProvider(),
    default_mode="optimize",
)

# Per-request overrides
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    headroom_output_buffer_tokens=8000,  # More room for long responses
    headroom_keep_turns=5,               # Protect last 5 turns
)

Note: The IntelligentContextConfig, ScoringWeights, and RollingWindowConfig classes are no longer part of Headroom. Context management is now handled automatically inside the pipeline (live-zone-only compression).