Proxy Server

The Headroom proxy is a standalone HTTP server that compresses all LLM traffic passing through it. Point any client at the proxy and get automatic context optimization.

Running a local OpenAI-compatible model? See Local LLM prefill benchmarking for a baseline-vs-optimized workflow that measures prompt-processing savings with the dashboard.

Starting the proxyCopy Anchor Link

# Basic usage
headroom proxy

# Deliberate public access, with the existing token protocol
HEADROOM_PROXY_TOKEN='replace-with-a-secret' headroom proxy --host 0.0.0.0 --port 8080
# Send `Authorization: Bearer replace-with-a-secret` or
# `X-Headroom-Proxy-Token: replace-with-a-secret` from the caller.

# With logging and budget
headroom proxy \
  --log-file /var/log/headroom.jsonl \
  --budget 100.0

Two telemetry switches exist. HEADROOM_TELEMETRY=on (or --telemetry) turns on local-only in-process usage stats that power your own /stats, /metrics, and dashboard — off by default, and nothing under this switch leaves the machine.

Separately, an anonymous usage beacon is on by default (opt-out). Once a session goes idle it POSTs a small summary of how compression behaved to a Headroom Labs collector.

Why it exists. Compression quality is the product. The beacon tells us when a release changes it — a ratio that regresses, a content type that starts getting skipped, a provider path that begins failing — across real workloads rather than only our own test corpus. That is the whole purpose; there is no other use.

What it sends. Counters and identifiers, never free text. The collector applies an allowlist server-side and drops everything else before storage (deploy/beacon/worker.js): token totals, compression ratios and rates, skip reasons, source and provider names, model ids, failure counts and status codes, and a routing summary. Alongside those, seven resource attributes: service name and version, install id, install mode, stack, OS type and CPU architecture. A typical event is about 2KB.

What it never sends. Your prompts, your completions, your code, file paths, file contents, environment variables, or anything else derived from request or response bodies. None of these are on the allowlist, so even if a future client sent one the collector would discard it.

The install id is a random UUID generated on first run and stored in your config directory. It is deliberately not derived from hostname, MAC address, or any hardware property — delete the file and you get a new one.

Disable it with HEADROOM_BEACON=off, HEADROOM_TELEMETRY=off (or --no-telemetry), the DO_NOT_TRACK=1 convention, or HEADROOM_OFFLINE=1 (which also disables update checks and license reporting). HEADROOM_BEACON=on keeps the beacon on even with local stats off. headroom telemetry names the setting that turned it off.

CLI optionsCopy Anchor Link

CoreCopy Anchor Link

Use --http-proxy or HEADROOM_HTTP_PROXY when only provider API traffic should go through a proxy:

headroom proxy --http-proxy http://proxy.internal:8080

Avoid setting process-wide variables such as HTTP_PROXY, HTTPS_PROXY, ALL_PROXY, or NO_PROXY for this use case. HTTPX reads those variables too, but Headroom also inherits them into tool executions, so they can proxy unrelated tool traffic.

Context managementCopy Anchor Link

Code-memory MCP (Serena)Copy Anchor Link

headroom wrap registers Serena as the code-memory MCP for semantic, symbol-level code navigation. Serena runs on demand via uvx — Headroom downloads and executes no binary of its own — and indexes the current project locally. Pass --code-memory none to register no code-memory MCP.

For Claude Code, the entry is written at local scope, which means it applies to the directory you wrapped and nothing else. That keeps a Serena that cannot start from breaking Claude Code sessions in unrelated projects. Pass --code-memory-scope user (or set HEADROOM_CODE_MEMORY_SCOPE=user) to register it machine-wide instead, which is how releases before this change behaved. If an earlier version already registered Serena machine-wide, the next headroom wrap claude moves it into the project scope for you — but only when Headroom installed that entry itself; a Serena you registered by hand is left alone.

Upgrading from tokensave? Earlier releases registered a tokensave MCP server (a downloaded Rust binary). tokensave has been retired in favour of Serena. On your next headroom wrap / headroom unwrap, Headroom removes the tokensave MCP entry it installed and switches you to Serena — nothing to migrate, since both are just indexes rebuilt from your source. The leftover tokensave binary in ~/.local/bin and any .tokensave/ folders are unused and safe to delete.

By default, the proxy uses the shared ContentRouter pipeline. It routes text, logs, JSON, code, images, and tool outputs through the currently enabled compressors and preserves reversible CCR markers where applicable.

# Maximize compression
headroom proxy --mode token

# Preserve provider prefix cache stability
headroom proxy --mode cache

Savings profilesCopy Anchor Link

HEADROOM_SAVINGS_PROFILE selects a named profile that seeds Headroom's whole compression posture — proxy mode, keep-ratio, which messages are compressed, and force_kompress — at proxy startup. It is read by headroom proxy and by the headroom wrap subprocesses. When unset, the default profile is coding.

An unrecognized HEADROOM_SAVINGS_PROFILE value logs a warning and falls back to coding (the same default an unset variable resolves to) — the proxy never fails to start over a bad profile name. balanced is used only as a last-resort fallback when the runtime has no coding entry at all (old-runtime/new-client version skew). See headroom/agent_savings.py for each profile's full set of knobs.

Because the default coding profile uses cache mode (and the proxy's own default mode is also cache), Headroom runs in cache mode out of the box. For a direct headroom proxy invocation, mode precedence is --mode > HEADROOM_MODE > the cache default. The selected HEADROOM_SAVINGS_PROFILE does not provide a third-level mode fallback after startup; wrapped launches may seed HEADROOM_MODE before starting the proxy. To run token mode, pass --mode token or set HEADROOM_MODE=token:

# Aggressive ~90% token-savings profile
HEADROOM_SAVINGS_PROFILE=agent-90 headroom proxy --port 8787

Optional featuresCopy Anchor Link

headroom proxy --memory
headroom proxy --learn --min-evidence 3
headroom proxy --codex-wire-debug
HEADROOM_COMPRESS_PASSTHROUGH=1 headroom proxy

Savings profilesCopy Anchor Link

The proxy uses a savings profile to control compression behavior — which messages get compressed, how aggressively, and whether to prioritize provider prefix-cache stability or raw savings. Only the env var survives across related tools (headroom wrap passes it to the proxy it launches).

# Switch to a different profile
HEADROOM_SAVINGS_PROFILE=agent-90 headroom proxy

Built-in profilesCopy Anchor Link

coding (default) — Optimizes for coding-agent workloads with Anthropic. Uses cache mode (proxy_mode="cache"): compresses only the newest delta in each turn so the provider's prefix-cache is never busted. User messages are compressed, system prompts preserved (hottest cache). protect_recent is 0 — in cache mode the delta already is the newest turn, so a turn-count guard would suppress the only thing cache mode compresses; byte-exact fidelity instead comes from protect_reads=True (file reads are never lossy-compressed). Lossless-first with lossy fallback; tool search and cross-turn dedup enabled. This is the profile that headroom wrap uses.

agent-90 — Forces ML-based (Kompress) compression with a 10% keep-ratio, ignoring the lossless path. Compresses both user and system messages. Designed for non-coding or cost-sensitive workloads where maximum compression is the goal.

balanced — Token-mode compression with a 30% keep-ratio. Uses the standard lossless pipeline (does not force Kompress). Protects 4 recent turns. A safe general-purpose profile.

general — Token-mode compression for non-coding conversations. No turn protection (protect_recent=0 — nothing code-positional to preserve) and does not compress user or system messages. Uses the standard lossless pipeline.

Profile defaults and overridesCopy Anchor Link

Profiles seed compression settings. For a direct headroom proxy invocation, explicit mode settings take precedence in this order: --mode (CLI) > HEADROOM_MODE (environment) > the cache default. The selected profile does not replace that final fallback, but wrapped launches may pre-seed HEADROOM_MODE. The coding profile defaults to cache, and an explicit --mode token still takes effect:

# Coding's default mode is cache
headroom proxy
# An explicit CLI mode overrides the profile default
headroom proxy --mode token
# An explicit environment mode also overrides the profile default
HEADROOM_MODE=token HEADROOM_SAVINGS_PROFILE=coding headroom proxy

You can also choose a profile whose default is token mode:

HEADROOM_SAVINGS_PROFILE=agent-90 headroom proxy --mode token

Extending a profile with env overridesCopy Anchor Link

Profile defaults are applied only when the corresponding env var is not already set. You can start from a named profile and override individual settings:

# Start from coding but force Kompress on
HEADROOM_SAVINGS_PROFILE=coding HEADROOM_FORCE_KOMPRESS=1 headroom proxy

# Start from balanced but lower the keep-ratio
HEADROOM_SAVINGS_PROFILE=balanced HEADROOM_TARGET_RATIO=0.15 headroom proxy

Custom profilesCopy Anchor Link

For permanent custom profiles, see the profile definitions in headroom/agent_savings.py. Each profile is an AgentSavingsProfile dataclass with fields for compression mode, target ratio, turn protection, and pipeline toggles.

Serving on a Unix socketCopy Anchor Link

--uds binds an AF_UNIX socket instead of a TCP port. Request handling is identical — only the transport changes.

headroom proxy --uds ~/.headroom/run/proxy.sock

Two reasons to prefer it over a loopback port:

• No port to collide with, and no port exposed. Access is governed by filesystem permissions rather than by anything listening on the network.
• Deployments where a port is awkward. Containers and systemd units can share a socket through a bind mount or a runtime directory without publishing a port.

Point any Unix-socket-capable client at the path. With curl:

curl --unix-socket ~/.headroom/run/proxy.sock http://localhost/health

Permissions and lifecycleCopy Anchor Link

A Unix socket carries no credentials of its own, so the mode of the directory holding it is the access-control boundary.

If the parent directory does not exist, Headroom creates it 0700. If it already exists, Headroom does not touch its permissions — something else owns that directory's policy, and silently tightening it could lock out whatever put it there. Instead it refuses to start when an existing parent is writable by other users and not sticky, since anyone on the host could then replace the socket. (A sticky directory such as /tmp is accepted: others may create their own entries there but cannot unlink or rename yours.)

On startup Headroom removes a stale socket left behind by a crashed process, but it refuses to start when:

• something is still listening on the path (two proxies would silently split traffic),
• the path exists and is not a socket (far more likely a typo'd argument pointing at real data than a leftover), or
• the path exceeds the platform's sun_path limit — 108 bytes on Linux, 104 on macOS and the BSDs. Keep it short; a long path fails inside bind() with an error that names nothing.

Configuration in depthCopy Anchor Link

Proxy behavior is set by three layers, each overriding the one before:

1. Savings profile (HEADROOM_SAVINGS_PROFILE) — seeds a whole posture (mode, keep-ratio, which roles get compressed, Kompress on/off). Default coding. See Savings profiles.
2. Environment variables — nearly every CLI flag has an HEADROOM_* twin, which is what you'll use in Docker, systemd, or CI.
3. CLI flags — the most explicit; they win over env and profile.

The tables below group the knobs by what they control. They aren't exhaustive (headroom proxy --help prints the full list), but they cover what real deployments actually touch. Unless noted, every option is off/unset by default and safe to ignore.

Compression tuningCopy Anchor Link

Fine-grained control over what gets compressed and how hard. Most users pick a profile instead and never touch these.

Kompress (ML compression)Copy Anchor Link

Kompress is the ModernBERT/ONNX compressor that ContentRouter falls back to for prose and unstructured text. It can run in-process or be offloaded to a hosted endpoint.

Reversible compression (CCR) and lossless modeCopy Anchor Link

By default Headroom stores originals so the model can recover them via headroom_retrieve. See Reversible Compression.

File-read handlingCopy Anchor Link

Coding agents re-read the same files repeatedly; these control how stale reads are handled without busting the prefix cache.

Reliability: timeouts, retries, limitsCopy Anchor Link

Tool search and MCPCopy Anchor Link

Defers large tool schemas so they don't sit in every request. See MCP.

Cost-aware model routingCopy Anchor Link

Rewrite the upstream model per request — for example, send small, tool-free calls to a cheaper model. Opt-in and off by default. Configure with HEADROOM_MODEL_ROUTER_ENABLED plus HEADROOM_MODEL_ROUTES; see Cost-aware model routing.

ObservabilityCopy Anchor Link

See Metrics for the Prometheus and Grafana setup.

Who can read the logsCopy Anchor Link

Headroom writes the runtime log (~/.headroom/logs/proxy-<port>.log) and, when --log-file is set, the JSONL request log. Both can hold request and response content — --log-messages bodies, wire-debug dumps, and CCR payload previews when they are enabled.

• Linux and macOS: every log file Headroom creates is 0600, owner-read/write only, and so is every rotated backup (proxy-8000.log.1, …). This is not conditional on which content switch is on. If the log path already exists as a symlink, Headroom refuses to log rather than write through it, and says so.
• Windows: Headroom does not restrict the log file. Read access on NTFS is decided by the file's ACL; chmod there only toggles the read-only attribute, and Python has no API for setting an ACL, so Headroom does not claim a protection it cannot establish. The log inherits the ACL of the directory it is written into, and the proxy logs a warning at startup saying so. Keep %USERPROFILE%\.headroom (or your chosen --log-file path) off shared volumes and out of directories with broadened permissions.

A log shipper or sidecar running as a different user or group cannot read the runtime log on Linux/macOS. Run the collector as the same user, or copy the file with a privileged job.

Security and networkingCopy Anchor Link

PerformanceCopy Anchor Link

API endpointsCopy Anchor Link

GET /healthCopy Anchor Link

curl http://localhost:8787/health

{
  "status": "healthy",
  "optimize": true,
  "stats": {
    "total_requests": 42,
    "tokens_saved": 15000,
    "savings_percent": 45.2
  }
}

GET /statsCopy Anchor Link

Live session statistics plus durable persistent_savings totals. Stored at ~/.headroom/proxy_savings.json (override with HEADROOM_SAVINGS_PATH).

curl http://localhost:8787/stats

GET /stats-historyCopy Anchor Link

Durable history with hourly, daily, weekly, and monthly rollups. Powers the /dashboard view.

curl http://localhost:8787/stats-history
curl "http://localhost:8787/stats-history?format=csv&series=weekly"

GET /metricsCopy Anchor Link

Prometheus-format metrics for monitoring.

curl http://localhost:8787/metrics

headroom_requests_total{mode="optimize"} 1234
headroom_tokens_saved_total 5678900
headroom_persistent_savings_tokens_saved_total 5678900
headroom_compression_ratio_bucket{le="0.5"} 890
headroom_latency_seconds_bucket{le="0.01"} 800
headroom_cache_hits_total 456

headroom_tokens_saved_total is the runtime counter for the current proxy process. Use headroom_persistent_savings_tokens_saved_total for durable lifetime savings that match /stats.persistent_savings.

POST /v1/messagesCopy Anchor Link

Anthropic API format. The proxy compresses messages, forwards to Anthropic, and returns the response.

POST /v1/chat/completionsCopy Anchor Link

OpenAI API format. The proxy compresses messages, forwards to OpenAI, and returns the response.

POST /v1/responsesCopy Anchor Link

OpenAI Responses API format. The proxy compresses input payloads where applicable, forwards the request, and returns the response.

For Codex-compatible clients, the proxy also accepts these alias paths and routes them through the same handler:

• POST /v1/codex/responses
• POST /backend-api/responses
• POST /backend-api/codex/responses

Matching WebSocket and subpath aliases are also supported for Codex flows.

Codex Live voice WebSocketCopy Anchor Link

The proxy relays Codex Live voice frames without parsing or transforming them. These paths use the same transparent transport:

• ws://localhost:8787/v1/live
• ws://localhost:8787/v1/codex/live
• ws://localhost:8787/backend-api/live
• ws://localhost:8787/backend-api/codex/live

Subscription authentication uses the derived ChatGPT backend path. API-key authentication preserves the selected OpenAI-compatible base URL and inbound path. The backend Live suffix defaults to /live and can be corrected with HEADROOM_CODEX_LIVE_WS_PATH if the upstream contract changes. The exact ChatGPT backend path is not confirmed by this proxy documentation.

POST /v1internal:streamGenerateContentCopy Anchor Link

Google Cloud Code Assist / Antigravity compatibility endpoint used by Pi-style google-gemini-cli and google-antigravity providers.

The proxy also accepts:

• POST /v1/v1internal:streamGenerateContent

Antigravity IDECopy Anchor Link

Google's Antigravity IDE can route its model traffic through the proxy:

headroom wrap antigravity          # Start proxy + print Antigravity setup instructions
headroom mcp install --agent antigravity   # Register the Headroom MCP server in Antigravity

wrap antigravity prints a custom OpenAI-compatible model-provider base URL to add in Antigravity's model-provider settings; Antigravity fetches the model list from GET /v1/models on its own. Native Cloud Code Assist traffic (google-antigravity clients) is accepted on POST /v1internal:streamGenerateContent and routed upstream like any other Cloud Code request.

MCP registration writes to Antigravity's global mcp_config.json under ~/.gemini/ (preferring the install's current location when one already exists) and refuses to overwrite a config file that does not parse as JSON.

POST /v1/compressCopy Anchor Link

Compression-only endpoint. Compresses messages and returns them without ever making a completion request to an LLM provider — no generation, no provider API key, no upstream chat call. Used by the TypeScript SDK, by LiteLLM's headroom guardrail, and by API gateways running Headroom as a sidecar.

Message formatCopy Anchor Link

The endpoint does no format conversion. Whatever shape you send in messages is the shape you get back, and both wire formats are compressed natively:

• OpenAI shape — role: "tool" messages with tool_call_id, assistant tool_calls
• Anthropic shape — content-block lists with tool_use / tool_result / thinking blocks

So an Anthropic-native caller does not need to convert to OpenAI format first. Block types, tool_use_ids and message order are all preserved.

model selects the tokenizer (per-model, from Headroom's tokenizer registry) and the context limit. Send the real model name — including gateway-prefixed forms like bedrock/anthropic.claude-3-5-sonnet or vertex_ai/claude-sonnet-4@20250514 — so token counts and compression aggressiveness are right.

RequestCopy Anchor Link

config fields:

config.mode values:

• unset (default) — marker-free. Emits no <<ccr:…>> retrieval markers and writes nothing to the CCR store, so you can forward the returned messages straight to a provider. This is the right mode for a gateway or guardrail that just swaps messages and forwards.
• ccr — emits CCR markers and writes to the store. Only for callers that also inject the headroom_retrieve tool and can reach /v1/retrieve (itself loopback-only). Markers are a dangling pointer for the model otherwise.
• lossy_inline (alias lossless_then_lossy) — runs the lossless byte/data fold first, then compresses the folded remainder. Marker-free.

ResponseCopy Anchor Link

{
  "messages": [{ "role": "user", "content": "..." }],
  "tokens_before": 15000,
  "tokens_after": 3500,
  "tokens_saved": 11500,
  "compression_ratio": 0.23,
  "transforms_applied": ["router:smart_crusher:0.35"],
  "transforms_summary": { "router:smart_crusher:0.35": 1 },
  "ccr_hashes": []
}

HeadersCopy Anchor Link

x-headroom-bypass: true (case-insensitive) skips compression entirely and echoes your messages back with zeroed metrics. The bypass and empty-messages responses omit transforms_summary.

Errors and fail-openCopy Anchor Link

Without a session, compression fails open on timeout: you get 200 with your original messages, zeroed metrics, plus compression_skipped: true and skip_reason: "compression_timeout". Always check compression_skipped if you need to know whether compression actually ran. In session mode the same timeout is a 503 instead — see below for why.

Requests are recorded under provider="compress" in /stats and /metrics.

Multi-turn usage: keeping the prefix cacheCopy Anchor Link

This is the single most important thing to get right, and the default is not safe for an agent loop.

When Headroom proxies a request itself it watches the provider's cache hit rate turn over turn and freezes the already-cached prefix. Without config.session_id, /v1/compress cannot do that: it sees one isolated call and has no idea what the provider already cached. You have two options — let Headroom keep the state (session mode, below) or keep it yourself with frozen_message_count, which the rest of this section covers.

The provider caches the bytes you forwarded. Compression changed those bytes, so your original messages and the ones the provider cached are no longer the same thing — and it is the forwarded version you have to keep reproducing. Send the pristine originals again next turn and the provider sees a different prefix and re-reads it from scratch. On Anthropic a cache read is ~90% cheaper than fresh input, so that can easily cost more than the compression saves.

Compression is also not uniform over a conversation: how hard a message is compressed depends partly on how far it now sits from the end, so an older tool result can fall outside the recent-read protection window as the conversation grows and be compressed harder than it was last turn. Another reason not to rely on re-compression reproducing earlier output.

Two rules:

1. Pass config.frozen_message_count — how many leading messages the provider has already cached.
2. Send back your own previous output, not the original messages. frozen_message_count returns those leading messages exactly as you passed them in — it pins whatever you hand it. Hand it pristine originals and you get pristine originals back, which is precisely the prefix the provider does not have.

# Keep what you FORWARDED, not what you started with.
forwarded: list[dict] = []

def next_turn(new_messages: list[dict]) -> list[dict]:
    body = {
        "messages": forwarded + new_messages,
        "model": "claude-sonnet-4-6",
        # Everything already forwarded is already cached upstream — pin it.
        "config": {"frozen_message_count": len(forwarded)},
    }
    result = requests.post(f"{proxy}/v1/compress", json=body).json()
    forwarded[:] = result["messages"]   # becomes next turn's frozen prefix
    return forwarded

Also for multi-turn callers:

• Leave config.mode unset. The default is marker-free, which is what a forward-only caller wants.
• Send the real model name so the tokenizer and context limit resolve correctly — including gateway-prefixed forms.
• protect_recent is not a substitute. It guards the newest messages; frozen_message_count guards the oldest, which is the cached end.

Session modeCopy Anchor Link

Pass config.session_id and Headroom keeps the conversation's replay state itself — the same per-session compression cache and prefix tracker the proxy path uses — so a gateway that owns routing can resend the raw conversation every turn and still get a byte-identical prefix back. Whatever this endpoint previously returned for the session is treated as the provider's cache contract: every already-returned message comes back unchanged, even where recompressing it would do better, and only the new tail is compressed. An explicit config.frozen_message_count still wins when it is larger.

{
  "messages": [{ "role": "user", "content": "..." }],
  "model": "claude-sonnet-4-6",
  "config": { "session_id": "conv-8f1c" }
}

The response gains a session object:

"session": { "id": "conv-8f1c", "frozen_message_count": 12, "cached_prefix_replayed": true }

What differs from the stateless contract:

• Forward the returned messages verbatim. They are what the next turn replays.
• compress_user_messages is refused (400). User-message rewrites are not byte-replayable across turns, so they would bust the prefix later.
• Timeouts and busy sessions are a 503, never fail-open. Handing back originals would leave the caller forwarding bytes the replay state does not know about, so a compression timeout answers 503 with error.type = "compression_timeout"; retry the turn and it lands on consistent state. Turns for one session are serialized under a per-session lock, and a concurrent turn for the same session that waits more than 10 seconds gets the same 503.
• State is in-process. The replay cache lives for HEADROOM_COMPRESSION_CACHE_TTL_SECONDS of idle time (3900 by default, floor 600); the tracker's usage state expires after 10 idle minutes. Multi-process deployments (gunicorn workers, replicas) must pin a session to one process — hash the upstream on the session id — or replay silently misses.
• HEADROOM_COMPRESS_SESSION_FROM_HEADER=1 lets the x-headroom-session-id header stand in for config.session_id. Off by default: gateways commonly stamp that header on all traffic, and a value shared across conversations (Claude Code subagents do this) would blend them into one replay state.

Relaying provider usage: POST /v1/usage. Session mode never sees the provider's response, so cache-hit attribution and savings accounting for the session come from this optional relay (same exposure rules as /v1/compress). Post the provider's usage block under the session id:

{
  "session_id": "conv-8f1c",
  "usage": { "cache_read_input_tokens": 11800, "cache_creation_input_tokens": 400 }
}

OpenAI-shaped usage is accepted too: prompt_tokens_details.cached_tokens or a flat cached_tokens is read as the cache-read count (OpenAI reports no write counter). A block carrying none of these fields is a 400 — it has no cache signal, and reading it as zeros would wipe the tracker's cached-prefix state.

The reply is { "session_id": "conv-8f1c", "frozen_message_count": 12, "applied": true }. applied: false with reason: "no_cache_signal" means the only present field was 0 (a fully cold turn can only be asserted with both fields present). 404 with error.type = "unknown_session" means the tracker's state for that session has expired — replay may still be active; only this telemetry landed nowhere. 503 with error.type = "session_busy" means a compress turn for the session is in flight; retry.

The relay is telemetry only. It never raises the frozen count, because a provider-confirmed count could freeze a message whose cache entry has since been evicted. Skipping it costs attribution, never correctness.

Gateway mode (two-half turn contract)Copy Anchor Link

An API gateway that owns routing (Kong, LiteLLM, a custom sidecar) never lets Headroom see the provider's response, which rules out every transform whose reload step needs it. Gateway mode splits one model turn into two HTTP calls: the request half (POST /v1/compress with a top-level gateway object) returns the complete provider body plus the obligations the gateway must redeem, and the response half (POST /v1/compress/response) takes the provider's answer and either finishes the turn or asks the gateway to call the provider again.

The rule behind it is the proxy's own no shrink without reload, moved to the API boundary: a transform whose reload step needs the response only runs when the gateway has said it can re-drive. A body without a gateway key never touches any of this — stateless and session behaviour are unchanged.

The gateway block. Presence is the switch: "gateway": {} is gateway mode with every default. Unknown keys are ignored (a newer plugin can talk to an older Headroom); a non-object block or a non-boolean field is a 400.

What changes on the request half. Every top-level field other than messages, model, token_budget, config and gateway passes through into the returned body: system, tools, temperature, stream, metadata, anything the provider accepts. tools is compacted deterministically (tool_schema_compaction in transforms_applied, plus tool_desc_compaction when HEADROOM_TOOL_DESC_MAX_CHARS is set), so identical tools come back byte-identical every turn. Two more request-only, streaming-safe levers run here exactly as they do on the proxy path:

• Tool-search deferral (Anthropic) is an extension's job on this path. OSS core does not mark tools defer_loading on /v1/compress; the headroom-tool-search extension's native tier does, through the turn-hook contract, when it is installed, enabled (HEADROOM_PROXY_EXTENSIONS=tool_search) and licensed. It applies the same primitives as the proxy path (12 or more Anthropic-shaped tools on a first-party Claude model, the same resident set, the same client-already-searching check, HEADROOM_TOOL_SEARCH=0 as the kill switch), is a pure function of the input tools so the tools prefix keeps prompt-caching turn over turn, and is named tool_search:native_deferral:<n>tools:<n>tok in transforms_applied and credited as tool_schema_tokens_saved in /stats. Without the extension the tools pass through compacted and headers is {}. Tool-search history repair stays in core regardless: once a transcript carries Anthropic's tool-search blocks, a later request whose tools cannot support them (a side request with a smaller tools array) has those blocks replaced in place by an inert placeholder text block, as on the proxy path (router:tool_search_repair:<n>blocks).
• Output steering. When the output shaper is on (HEADROOM_OUTPUT_SHAPER=1 or the proxy_output_shaper rollout; off in mode="cache" for steering), the byte-stable verbosity block is appended to the tail of system (Anthropic) or the last system/developer message (OpenAI chat), the conversation-stable HEADROOM_OUTPUT_HOLDOUT arm is honoured, and the output_shaper:stratum:… / output_shaper:control:… label rides transforms_applied — it is what turns the response half's relayed output_tokens into output savings in /stats. The block is identical every turn, so the prefix still caches; it is applied to the returned body only, never to the session snapshot, so it is re-appended once per turn rather than accumulating.

Both fail open: any error leaves the body as built and headers empty.

Turn hooks registered by proxy extensions run here as they do on the proxy path. Request hooks run stream-safe-only unless can_redrive and session_affinity are both true — a hook that must see the response is skipped when the gateway cannot come back with one. Hook savings show up as turn_hook and turn_hook:tools:<n>tok. In session mode hooks run after the replay overlay and before the snapshot, so a hook that is not a pure function of its input busts the prefix exactly as it would on the proxy.

The response keeps every field above and adds six:

{
  "messages": [{ "role": "user", "content": "..." }],
  "tokens_before": 15000,
  "tokens_after": 3500,
  "body": {
    "model": "gpt-4o",
    "messages": [{ "role": "user", "content": "..." }],
    "tools": [{ "type": "function", "function": { "name": "search", "parameters": {} } }],
    "temperature": 0.2
  },
  "turn_id": "3d2f6c0e0b0a4e1a9c1b2f3d4e5f6a7b",
  "route": { "model": "gpt-4o", "provider": "openai", "service_tier": null, "reason": "" },
  "obligations": ["relay_usage"],
  "gateway": { "can_redrive": false, "can_relay_response": true, "session_affinity": true },
  "headers": {}
}

Fail-open answers (bypass header, empty messages, stateless timeout) carry the same six keys with your original body, obligations: [] and headers: {}.

When relay_usage is owed, the turn's /stats record is deferred until the response half arrives with billed input and output, cache counts, provider status and latency; a turn that never comes back is recorded as-is when it expires. Pending turns live HEADROOM_GATEWAY_TURN_TTL_SECONDS (default 120, refreshed on every redrive), at most HEADROOM_GATEWAY_MAX_PENDING_TURNS at a time (default 10000, oldest evicted first), and a turn may re-drive at most HEADROOM_GATEWAY_MAX_REDRIVES times (default 8).

config.mode: "ccr" on this contract. When markers were inserted and re-driving is allowed, Headroom injects the headroom_retrieve tool into body.tools (ccr_tool_injected in transforms_applied), adds the redrive obligation, and the response half answers the model's retrieval calls itself — the gateway just keeps re-driving. Without can_redrive, ccr mode behaves as it does today: the markers are yours to resolve.

POST /v1/compress/responseCopy Anchor Link

The response half. Same exposure rules as /v1/compress: loopback-only unless HEADROOM_COMPRESS_ALLOW_REMOTE=1, and HEADROOM_PROXY_TOKEN applies.

Accepted usage shapes — the whole provider response body works too, since a nested usage key is descended into once:

• Anthropic: input_tokens, output_tokens, cache_read_input_tokens, cache_creation_input_tokens
• OpenAI chat: prompt_tokens, completion_tokens, prompt_tokens_details.cached_tokens
• OpenAI Responses: input_tokens, output_tokens, input_tokens_details.cached_tokens
• Kong's flattened ai.proxy log shape: prompt_tokens, completion_tokens, top-level cached_tokens

Booleans and negative counts are a 400. Usage with no cache field is accepted but not applied to the session tracker; the apply rule is /v1/usage's (a positive count, or both cache fields present at 0). Across re-drive rounds the billed counters are summed and the cache fields come from the latest round that carried them.

There are two answers. done ends the turn:

{
  "action": "done",
  "turn_id": "3d2f6c0e0b0a4e1a9c1b2f3d4e5f6a7b",
  "response": null,
  "frozen_message_count": 12,
  "usage_applied": true,
  "rounds": 0,
  "billed_usage": { "input_tokens": 3120, "output_tokens": 212, "cache_read_input_tokens": 20414 }
}

response is what to send to the client. On a turn with no re-drive it is the hook's replacement when there is one, otherwise null and you forward the provider response you already hold. After one or more re-drives it is never null: it is the hook's replacement or the latest provider response, with usage set to the whole turn's bill. frozen_message_count is the session's count after the usage was applied (null for a stateless turn). usage_applied is false when the usage carried no cache signal, the session had expired, or its lock was busy — never an error, since the turn is finishing and there is nothing to retry with. billed_usage is what the provider billed for the whole turn: every counter, cache reads and writes included, summed across all rounds, in Anthropic keys (null when no usage was relayed). That total is the re-driven response's usage, since the client made one call. After done the turn is gone.

redrive asks for another provider call:

{
  "action": "redrive",
  "turn_id": "3d2f6c0e0b0a4e1a9c1b2f3d4e5f6a7b",
  "request": { "model": "gpt-4o", "messages": [], "tools": [] },
  "round": 1
}

Send request to the provider exactly as given — it is the provider body with the hook's messages and possibly reloaded tools — and post the provider's JSON back under the same turn_id. Past HEADROOM_GATEWAY_MAX_REDRIVES the turn ends with done and the latest provider response. Hook failures never fail the turn either: the answer is done, with the latest provider response after a re-drive, or response: null before one, and the gateway forwards what it gets.

Kong pluginCopy Anchor Link

kong-plugin-headroom is a Kong Gateway 3.9 plugin, in its own repository and installable with luarocks install kong-plugin-headroom or as a Konnect custom plugin, that implements both halves: it sends every chat request through /v1/compress with config.session_id and a gateway block, swaps body in, relays usage from the log phase, and runs the re-drive loop in access when a redrive obligation comes back. Install steps, configuration and limits are in its README. Headroom's docker end-to-end test (tests/gateway/test_kong_integration.py, HEADROOM_KONG_E2E=1) expects a checkout of it next to this repository or at HEADROOM_KONG_PLUGIN_DIR.

The contract is itself an extension. Core exposes a compress-turn seam (headroom.proxy.compress_turn): an extension registers a contract with register_compress_turn_extension, and per request it may claim the body (begin), prepare executor-side work (prepare), rewrite the outbound messages at the one safe moment in session mode (transform, after the prefix replay and before the tracker records what was returned), add response fields and transform labels (finish), take ownership of the outcome (commit), and answer the fail-open cases (fail_open_fields). The gateway turn contract described above is installed through that seam by headroom.proxy.gateway_extension (on by default; HEADROOM_GATEWAY_CONTRACT=0 leaves it out, after which a gateway block is ignored and the endpoint answers in legacy shape). A third-party contract's install(app, config) finds the /v1/compress exposure policy on app.state.compress_route_dependencies for any route it adds.

Agent wrappingCopy Anchor Link

Use headroom wrap to launch supported CLI agents through the local proxy:

# Claude Code
headroom wrap claude

# Claude Code extension in VS Code (configures settings, then starts the proxy)
headroom wrap vscode-claude

# OpenAI Codex
headroom wrap codex

# Aider
headroom wrap aider

# Cursor (starts the proxy and prints settings to paste into Cursor)
headroom wrap cursor

# Grok Build (updates ~/.grok/config.toml and starts the proxy)
headroom wrap grok-build

Wrapping can reuse an existing proxy instead of starting a new process. Its startup-only settings do not change when you export different values in the calling shell. wrap warns when an explicit HEADROOM_MODE, HEADROOM_MIN_TOKENS, or HEADROOM_EXCLUDE_TOOLS disagrees with the settings reported by that proxy. It does not restart a shared proxy or interrupt other clients. Restart the proxy with the desired settings, or use --port to start a separate instance. Tool exclusions are compared case-insensitively using the effective router exclusions, including built-ins and tools protected by HEADROOM_PROTECT_TOOL_RESULTS. Older proxies that do not report a setting cannot be compared for that setting.

headroom wrap codex keeps routing scoped to the Codex process it launches: it passes the proxy URL as a --config override and never writes it to ~/.codex/config.toml. Codex runs any session started with command-line overrides in embedded mode, so a wrapped session does not use Codex's shared background server and Codex prints "Running without the shared background server". If you want the shared server, write the proxy routing to Codex's own config instead, then start codex directly:

headroom init codex     # this project's .codex/config.toml (trusted projects)
headroom init -g codex  # ~/.codex/config.toml, for every project

That routing is persistent: it applies to every Codex session in that scope until you remove it. headroom unwrap codex removes the -g routing; for a project, delete the # --- Headroom init provider --- block from .codex/config.toml. Unwrap removes only the routing: the Headroom SessionStart hook that init adds to Codex's hooks.json stays, and still starts a proxy when a Codex session begins until you delete it. On Windows, nothing starts the proxy for this routing, so Codex cannot connect while the proxy is down; use headroom install apply if you need a supervised proxy.

Cursor reads model endpoints from its settings UI, so headroom wrap cursor does not rewrite Cursor configuration or launch the app. After it starts the proxy, copy the printed base URL into Cursor's model settings.

Grok Build reads model endpoints from ~/.grok/config.toml. headroom wrap grok-build injects or updates [model.grok-build] base_url to point at the local proxy, then run grok from the same project directory. See Grok Build Integration.

The official Claude Code extension reads Claude Code's user settings rather than the terminal environment. Use headroom wrap vscode-claude, reload VS Code after the first run, and keep the wrapper running. See the VS Code Claude Code guide for verification and undo steps.

For environment-driven clients, you can also set the base URL manually:

# Claude Code
ANTHROPIC_BASE_URL=http://localhost:8787 claude

# Any OpenAI-compatible CLI client that reads OPENAI_BASE_URL
OPENAI_BASE_URL=http://localhost:8787/v1 your-client

Cloud providersCopy Anchor Link

# AWS Bedrock
headroom proxy --backend bedrock --region us-east-1

# Google Vertex AI
headroom proxy --backend vertex_ai --region us-central1

# Azure OpenAI
headroom proxy --backend azure

# OpenRouter (400+ models)
OPENROUTER_API_KEY=sk-or-... headroom proxy --backend openrouter

Google Vertex AICopy Anchor Link

--backend vertex_ai delegates to LiteLLM, which brings two requirements that are easy to miss:

1. Install the Vertex SDK. google-cloud-aiplatform is not included in any Headroom extra ([proxy], [all], …) or Docker image variant, so install it alongside Headroom:

pip install "headroom-ai[proxy]" "google-cloud-aiplatform>=1.38"

Without it, the first Vertex request fails with litellm.BadRequestError: … vertexai import failed … No module named 'vertexai'.

2. Set the LiteLLM project/location variables. LiteLLM reads the GCP project and region from VERTEXAI_PROJECT / VERTEXAI_LOCATION — these are not the standard Google Cloud variables (GOOGLE_CLOUD_PROJECT / GOOGLE_CLOUD_LOCATION) used by gcloud and current Google SDKs:

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json  # or use ADC
export VERTEXAI_PROJECT=<your-gcp-project>
export VERTEXAI_LOCATION=us-central1

headroom proxy --backend vertex_ai --region us-central1

Backend name aliases. vertex_ai, vertex, google-vertex, googlevertex, litellm-vertex, and litellm-vertex_ai are all normalized to the same LiteLLM-backed vertex_ai backend — CLI help text and older docs use these spellings interchangeably.

Running Claude Code against Claude models on Vertex? See Claude Code on Vertex AI for the recommended native Vertex-mode flow that reuses Claude Code's own GCP auth.

Native Vertex passthrough routesCopy Anchor Link

Separately from --backend vertex_ai, the proxy always registers routes that mirror Vertex's native REST shape verbatim — no backend flag needed:

/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:generateContent
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:streamGenerateContent
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:countTokens
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:rawPredict
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:streamRawPredict

Requests with publisher=google (Gemini models) go through Headroom's full Gemini optimization handler; publisher=anthropic (Claude on Vertex) routes through the same LiteLLM-Vertex path as --backend vertex_ai. Any client that already speaks the native Vertex REST API can simply point its endpoint at the proxy — this is the mechanism headroom wrap claude uses in Vertex mode.

Bedrock via a local gatewayCopy Anchor Link

--backend bedrock accepts Anthropic input (/v1/messages) and re-signs to AWS. Some setups are the other way around: the client already speaks Bedrock (e.g. Claude Code with CLAUDE_CODE_USE_BEDROCK=1, or any AWS SDK pointed at a custom endpoint), sending POST /model/{id}/invoke to a local gateway that re-signs and forwards to AWS (LiteLLM, LocalStack, a corporate Bedrock proxy).

--bedrock-api-url lets Headroom sit in that chain. It registers passthrough routes for /model/{id}/invoke and /model/{id}/invoke-with-response-stream, compresses the request body with the same pipeline as /v1/messages, and forwards to the gateway:

headroom proxy --bedrock-api-url http://127.0.0.1:4000
# then point the client's Bedrock endpoint at Headroom:
AWS_ENDPOINT_URL_BEDROCK_RUNTIME=http://127.0.0.1:8787 your-bedrock-client

The routes are registered only when --bedrock-api-url (or BEDROCK_TARGET_API_URL) is set — otherwise Bedrock requests fall through unchanged.

Environment variablesCopy Anchor Link

# For deliberate public access, configure the existing token and send it from callers.
export HEADROOM_HOST=0.0.0.0
export HEADROOM_PROXY_TOKEN='replace-with-a-secret'
export HEADROOM_PORT=8787
export HEADROOM_BUDGET=100.0

# Route OpenAI passthrough requests to a custom endpoint
export OPENAI_TARGET_API_URL=https://custom.openai.endpoint.com

# Route Anthropic passthrough requests to a custom endpoint
export ANTHROPIC_TARGET_API_URL=https://litellm.company.internal

# Compress Bedrock InvokeModel traffic, forwarding to a re-signing gateway
export BEDROCK_TARGET_API_URL=http://127.0.0.1:4000

headroom proxy

Production deploymentCopy Anchor Link

gunicornCopy Anchor Link

pip install gunicorn

gunicorn "headroom.proxy.server:create_app()" \
  --workers 4 \
  --bind 0.0.0.0:8787 \
  --worker-class uvicorn.workers.UvicornWorker
# Callers must send `Authorization: Bearer replace-with-a-secret` or
# `X-Headroom-Proxy-Token: replace-with-a-secret`.

headroom.proxy.server has no module-level app object — it exposes a create_app(config: ProxyConfig | None = None) -> FastAPI factory, so gunicorn's import string must call it.

DockerCopy Anchor Link

FROM python:3.11-slim
RUN apt-get update && apt-get install -y --no-install-recommends build-essential \
    && pip install "headroom-ai[proxy]" \
    && apt-get purge -y build-essential && apt-get autoremove -y \
    && rm -rf /var/lib/apt/lists/*
EXPOSE 8787
CMD ["headroom", "proxy", "--host", "0.0.0.0", "--port", "8787"]

Run this image with an explicit token and a deliberate publication choice:

docker run --rm -p 127.0.0.1:8787:8787 \
  -e HEADROOM_PROXY_TOKEN='replace-with-a-secret' \
  headroom-proxy

Callers must send Authorization: Bearer replace-with-a-secret or X-Headroom-Proxy-Token: replace-with-a-secret. For network access, replace the host-side 127.0.0.1 with an intentional public address and keep the token configured.