Architecture

Headroom sits between your application and the LLM provider. It intercepts the request, compresses the parts that carry the most redundant tokens — tool outputs, file reads, logs, search results — and forwards the optimized request upstream. The provider's response is returned unchanged.

High-level flow

+---------------------------------------------------------------+
|                       YOUR APPLICATION                        |
+---------------------------------------------------------------+
                               |
                               v
+---------------------------------------------------------------+
|                           HEADROOM                            |
|    Proxy (FastAPI)   ·   Python compress()   ·   TS compress() |
|                               |                               |
|                               v                               |
|        Transform pipeline  ──▶  ContentRouter                 |
|      (detect content type, route to one compressor)           |
|                               |                               |
|                               v                               |
|             Backend  (direct · LiteLLM · any-llm)             |
+---------------------------------------------------------------+
                               |
                               v
+---------------------------------------------------------------+
|         OPENAI · ANTHROPIC · GOOGLE · BEDROCK · 100+          |
+---------------------------------------------------------------+

Entry points

Headroom can be used three ways, all feeding the same compression pipeline:

In proxy mode the server is a FastAPI app with per-provider handlers (Anthropic, OpenAI, Gemini, Bedrock) that each run the same compression pipeline before forwarding through the selected backend.

The compression pipeline

The proxy assembles a small, ordered pipeline. Every transform is independent, safe to skip, and fails open — on any error it returns the content unchanged and the request still goes through.

1. Tool-result interceptor (canary opt-in) — light structural interceptors such as ast-grep Read outlining. Requires HEADROOM_ROLLOUT_CHANNEL=canary plus --intercept-tool-results.
2. CacheAligner (off by default) — a detector that reports dynamic-prefix drift (dates, UUIDs, session tokens). It never mutates, moves, or rewrites content. It is disabled by default and hard-disabled inside the proxy; it exists to surface prefix-stability metrics, not to change your messages.
3. ContentRouter — the workhorse that does essentially all of the compression. See below.

The pipeline never drops or reorders messages. Earlier versions shipped a "context manager" stage (rolling-window / intelligent-context scoring) that deleted old turns to make the request fit the context window. That stage was removed — the RollingWindowConfig, IntelligentContextConfig, and ScoringWeights classes no longer exist. Headroom now does live-zone-only compression: it compresses content in place and leaves the message list intact. See Context Management.

ContentRouter

ContentRouter detects the type of each content block and dispatches it to exactly one compressor:

To avoid recompressing the same content, ContentRouter keeps a two-tier, TTL-bounded cache: a skip set of content already known not to compress, and a result cache of previously compressed output. Default TTL is 30 minutes.

Rust core

The heaviest compressors run in a native Rust extension — headroom._core, built with PyO3 — that the proxy loads at startup. SmartCrusher and the search/log/diff compressors and content detection are Rust-backed; the Python classes you import are thin, API-compatible shims over them. The ML fallback (Kompress, a ModernBERT token compressor) runs separately through ONNX Runtime, either locally or offloaded to a remote endpoint (see Text & Logs).

Compression modes and provider caches

Headroom runs in one of two modes (--mode, default cache):

• cache mode (default) — compresses only the newest delta in each turn and forwards prior turns byte-faithfully, so the provider's prefix cache is never invalidated mid-conversation. Best for coding agents and any long, multi-turn session.
• token mode — prioritizes raw token removal and may recompress earlier turns, trading some cache stability for maximum savings.

Because prefix caching is where most of the cost savings live on multi-turn workloads, keeping the stable prefix intact matters. What each provider's cache buys you:

See Cache Optimization and Savings Profiles for how the two modes interact with the built-in profiles.

CCR: Compress-Cache-Retrieve

Compression is reversible. When ContentRouter compresses a tool output, the original is stored in a local Compress-Cache-Retrieve (CCR) store. If the model needs the full data, it calls a headroom_retrieve tool and gets the original back.

Compress:  1000 items  ->  15 items   (original stored in CCR)
Cache:     hash-indexed local SQLite store
Retrieve:  model calls headroom_retrieve("abc123")  ->  original 1000 items

CCR is on by default. Disable the markers and the injected retrieval tool with --no-ccr, or run a marker-free, format-native lossless mode with --lossless. See Reversible Compression (CCR).

TOIN: Tool Output Intelligence Network

TOIN learns which fields matter for a given tool over repeated calls — which items get retrieved, which fields carry signal — and feeds that back into SmartCrusher's importance scoring so compression gets sharper for the tools you actually use.

TOIN is local and observation-only: it aggregates statistics on your own machine (or your own proxy instance). Nothing about your tools or traffic is shared across users or sent off the box. For a brand-new tool type it falls back to statistical heuristics and improves as it observes more calls.

What Headroom does not rewrite

• Your prompt text — the natural-language instructions you write are preserved. Compression targets bulk content blocks (tool outputs, file reads, logs), not your intent.
• System prompts — preserved by default so the hottest part of the prefix cache stays stable. A savings profile can opt into compacting them.
• Code — passes through unchanged unless AST-based code compression is explicitly enabled (off by default).
• Model responses — returned from the provider unchanged.
• Short content — blocks below the minimum-token threshold pass through (overhead would exceed savings).