Proxy Server

The Headroom proxy is a standalone HTTP server that compresses all LLM traffic passing through it. Point any client at the proxy and get automatic context optimization.

Running a local OpenAI-compatible model? See Local LLM prefill benchmarking for a baseline-vs-optimized workflow that measures prompt-processing savings with the dashboard.

Starting the proxy

# Basic usage
headroom proxy

# Custom host and port
headroom proxy --host 0.0.0.0 --port 8080

# With logging and budget
headroom proxy \
  --log-file /var/log/headroom.jsonl \
  --budget 100.0

Telemetry is local-only and off by default. HEADROOM_TELEMETRY=on (or --telemetry) turns on in-process usage stats that power your own /stats, /metrics, and dashboard — nothing is sent to Headroom Labs. (The anonymous aggregate beacon that older versions shipped has been removed from the code.)

CLI options

Core

Use --http-proxy or HEADROOM_HTTP_PROXY when only provider API traffic should go through a proxy:

headroom proxy --http-proxy http://proxy.internal:8080

Avoid setting process-wide variables such as HTTP_PROXY, HTTPS_PROXY, ALL_PROXY, or NO_PROXY for this use case. HTTPX reads those variables too, but Headroom also inherits them into tool executions, so they can proxy unrelated tool traffic.

Context management

Code-memory MCP (Serena)

headroom wrap registers Serena as the code-memory MCP for semantic, symbol-level code navigation. Serena runs on demand via uvx — Headroom downloads and executes no binary of its own — and indexes the current project locally. Pass --code-memory none to register no code-memory MCP.

Upgrading from tokensave? Earlier releases registered a tokensave MCP server (a downloaded Rust binary). tokensave has been retired in favour of Serena. On your next headroom wrap / headroom unwrap, Headroom removes the tokensave MCP entry it installed and switches you to Serena — nothing to migrate, since both are just indexes rebuilt from your source. The leftover tokensave binary in ~/.local/bin and any .tokensave/ folders are unused and safe to delete.

By default, the proxy uses the shared ContentRouter pipeline. It routes text, logs, JSON, code, images, and tool outputs through the currently enabled compressors and preserves reversible CCR markers where applicable.

# Maximize compression
headroom proxy --mode token

# Preserve provider prefix cache stability
headroom proxy --mode cache

Savings profiles

HEADROOM_SAVINGS_PROFILE selects a named profile that seeds Headroom's whole compression posture — proxy mode, keep-ratio, which messages are compressed, and force_kompress — at proxy startup. It is read by headroom proxy and by the headroom wrap subprocesses. When unset, the default profile is coding.

An unrecognized HEADROOM_SAVINGS_PROFILE value logs a warning and falls back to balanced — the proxy never fails to start over a bad profile name. See headroom/agent_savings.py for each profile's full set of knobs.

Because the default coding profile uses cache mode (and the proxy's own default mode is also cache), Headroom runs in cache mode out of the box. Mode precedence: an explicit --mode wins, otherwise HEADROOM_MODE (which a profile seeds), otherwise the cache default. To run token mode, pass --mode token or choose a token-mode profile:

# Aggressive ~90% token-savings profile
HEADROOM_SAVINGS_PROFILE=agent-90 headroom proxy --port 8787

Optional features

headroom proxy --memory
headroom proxy --learn --min-evidence 3
headroom proxy --codex-wire-debug
headroom proxy --compress-passthrough

Savings profiles

The proxy uses a savings profile to control compression behavior — which messages get compressed, how aggressively, and whether to prioritize provider prefix-cache stability or raw savings. Only the env var survives across related tools (headroom wrap passes it to the proxy it launches).

# Switch to a different profile
HEADROOM_SAVINGS_PROFILE=agent-90 headroom proxy

Built-in profiles

coding (default) — Optimizes for coding-agent workloads with Anthropic. Uses cache mode (proxy_mode="cache"): compresses only the newest delta in each turn so the provider's prefix-cache is never busted. User messages are compressed, system prompts preserved (hottest cache). Protects the 2 most recent turns verbatim. Lossless-first with lossy fallback; tool search and cross-turn dedup enabled. This is the profile that headroom wrap uses.

agent-90 — Forces ML-based (Kompress) compression with a 10% keep-ratio, ignoring the lossless path. Compresses both user and system messages. Designed for non-coding or cost-sensitive workloads where maximum compression is the goal.

balanced — Token-mode compression with a 30% keep-ratio. Uses the standard lossless pipeline (does not force Kompress). Protects 4 recent turns. A safe general-purpose profile.

general — Token-mode compression for non-coding conversations. No turn protection (protect_recent=0 — nothing code-positional to preserve) and does not compress user or system messages. Uses the standard lossless pipeline.

Profiles override CLI flags

A profile's proxy_mode setting overrides the --mode flag. The coding profile sets proxy_mode="cache", so --mode token has no effect when coding is active:

# These are equivalent — coding's cache mode always wins
headroom proxy
headroom proxy --mode token          # --mode token is silently overridden

To run in token mode, switch to a profile that uses it:

HEADROOM_SAVINGS_PROFILE=agent-90 headroom proxy --mode token

Extending a profile with env overrides

Profile defaults are applied only when the corresponding env var is not already set. You can start from a named profile and override individual settings:

# Start from coding but force Kompress on
HEADROOM_SAVINGS_PROFILE=coding HEADROOM_FORCE_KOMPRESS=1 headroom proxy

# Start from balanced but lower the keep-ratio
HEADROOM_SAVINGS_PROFILE=balanced HEADROOM_TARGET_RATIO=0.15 headroom proxy

Custom profiles

For permanent custom profiles, see the profile definitions in headroom/agent_savings.py. Each profile is an AgentSavingsProfile dataclass with fields for compression mode, target ratio, turn protection, and pipeline toggles.

Configuration in depth

Proxy behavior is set by three layers, each overriding the one before:

1. Savings profile (HEADROOM_SAVINGS_PROFILE) — seeds a whole posture (mode, keep-ratio, which roles get compressed, Kompress on/off). Default coding. See Savings profiles.
2. Environment variables — nearly every CLI flag has an HEADROOM_* twin, which is what you'll use in Docker, systemd, or CI.
3. CLI flags — the most explicit; they win over env and profile.

The tables below group the knobs by what they control. They aren't exhaustive (headroom proxy --help prints the full list), but they cover what real deployments actually touch. Unless noted, every option is off/unset by default and safe to ignore.

Compression tuning

Fine-grained control over what gets compressed and how hard. Most users pick a profile instead and never touch these.

Kompress (ML compression)

Kompress is the ModernBERT/ONNX compressor that ContentRouter falls back to for prose and unstructured text. It can run in-process or be offloaded to a hosted endpoint.

Reversible compression (CCR) and lossless mode

By default Headroom stores originals so the model can recover them via headroom_retrieve. See Reversible Compression.

File-read handling

Coding agents re-read the same files repeatedly; these control how stale reads are handled without busting the prefix cache.

Reliability: timeouts, retries, limits

Tool search and MCP

Defers large tool schemas so they don't sit in every request. See MCP.

Cost-aware model routing

Rewrite the upstream model per request — for example, send small, tool-free calls to a cheaper model. Opt-in and off by default. Configure with HEADROOM_MODEL_ROUTER_ENABLED plus HEADROOM_MODEL_ROUTES; see Cost-aware model routing.

Observability

See Metrics for the Prometheus and Grafana setup.

Security and networking

Performance

API endpoints

GET /health

curl http://localhost:8787/health

{
  "status": "healthy",
  "optimize": true,
  "stats": {
    "total_requests": 42,
    "tokens_saved": 15000,
    "savings_percent": 45.2
  }
}

GET /stats

Live session statistics plus durable persistent_savings totals. Stored at ~/.headroom/proxy_savings.json (override with HEADROOM_SAVINGS_PATH).

curl http://localhost:8787/stats

GET /stats-history

Durable history with hourly, daily, weekly, and monthly rollups. Powers the /dashboard view.

curl http://localhost:8787/stats-history
curl "http://localhost:8787/stats-history?format=csv&series=weekly"

GET /metrics

Prometheus-format metrics for monitoring.

curl http://localhost:8787/metrics

headroom_requests_total{mode="optimize"} 1234
headroom_tokens_saved_total 5678900
headroom_persistent_savings_tokens_saved_total 5678900
headroom_compression_ratio_bucket{le="0.5"} 890
headroom_latency_seconds_bucket{le="0.01"} 800
headroom_cache_hits_total 456

headroom_tokens_saved_total is the runtime counter for the current proxy process. Use headroom_persistent_savings_tokens_saved_total for durable lifetime savings that match /stats.persistent_savings.

POST /v1/messages

Anthropic API format. The proxy compresses messages, forwards to Anthropic, and returns the response.

POST /v1/chat/completions

OpenAI API format. The proxy compresses messages, forwards to OpenAI, and returns the response.

POST /v1/responses

OpenAI Responses API format. The proxy compresses input payloads where applicable, forwards the request, and returns the response.

For Codex-compatible clients, the proxy also accepts these alias paths and routes them through the same handler:

• POST /v1/codex/responses
• POST /backend-api/responses
• POST /backend-api/codex/responses

Matching WebSocket and subpath aliases are also supported for Codex flows.

Codex Live voice WebSocket

The proxy relays Codex Live voice frames without parsing or transforming them. These paths use the same transparent transport:

• ws://localhost:8787/v1/live
• ws://localhost:8787/v1/codex/live
• ws://localhost:8787/backend-api/live
• ws://localhost:8787/backend-api/codex/live

Subscription authentication uses the derived ChatGPT backend path. API-key authentication preserves the selected OpenAI-compatible base URL and inbound path. The backend Live suffix defaults to /live and can be corrected with HEADROOM_CODEX_LIVE_WS_PATH if the upstream contract changes. The exact ChatGPT backend path is not confirmed by this proxy documentation.

POST /v1internal:streamGenerateContent

Google Cloud Code Assist / Antigravity compatibility endpoint used by Pi-style google-gemini-cli and google-antigravity providers.

The proxy also accepts:

• POST /v1/v1internal:streamGenerateContent

POST /v1/compress

Compression-only endpoint. Compresses messages and returns them without ever making a completion request to an LLM provider — no generation, no provider API key, no upstream chat call. Used by the TypeScript SDK, by LiteLLM's headroom guardrail, and by API gateways running Headroom as a sidecar.

Message format

The endpoint does no format conversion. Whatever shape you send in messages is the shape you get back, and both wire formats are compressed natively:

• OpenAI shape — role: "tool" messages with tool_call_id, assistant tool_calls
• Anthropic shape — content-block lists with tool_use / tool_result / thinking blocks

So an Anthropic-native caller does not need to convert to OpenAI format first. Block types, tool_use_ids and message order are all preserved.

model selects the tokenizer (per-model, from Headroom's tokenizer registry) and the context limit. Send the real model name — including gateway-prefixed forms like bedrock/anthropic.claude-3-5-sonnet or vertex_ai/claude-sonnet-4@20250514 — so token counts and compression aggressiveness are right.

Request

config fields:

config.mode values:

• unset (default) — marker-free. Emits no <<ccr:…>> retrieval markers and writes nothing to the CCR store, so you can forward the returned messages straight to a provider. This is the right mode for a gateway or guardrail that just swaps messages and forwards.
• ccr — emits CCR markers and writes to the store. Only for callers that also inject the headroom_retrieve tool and can reach /v1/retrieve (itself loopback-only). Markers are a dangling pointer for the model otherwise.
• lossy_inline (alias lossless_then_lossy) — runs the lossless byte/data fold first, then compresses the folded remainder. Marker-free.

Response

{
  "messages": [{ "role": "user", "content": "..." }],
  "tokens_before": 15000,
  "tokens_after": 3500,
  "tokens_saved": 11500,
  "compression_ratio": 0.23,
  "transforms_applied": ["router:smart_crusher:0.35"],
  "transforms_summary": { "router:smart_crusher:0.35": 1 },
  "ccr_hashes": []
}

Headers

x-headroom-bypass: true (case-insensitive) skips compression entirely and echoes your messages back with zeroed metrics. The bypass and empty-messages responses omit transforms_summary.

Errors and fail-open

Compression fails open on timeout: you get 200 with your original messages, zeroed metrics, plus compression_skipped: true and skip_reason: "compression_timeout". Always check compression_skipped if you need to know whether compression actually ran.

Requests are recorded under provider="compress" in /stats and /metrics.

Multi-turn usage: keeping the prefix cache

This is the single most important thing to get right, and the default is not safe for an agent loop.

When Headroom proxies a request itself it watches the provider's cache hit rate turn over turn and freezes the already-cached prefix. /v1/compress cannot do that — it is stateless. It sees one isolated call and has no idea what the provider already cached.

The provider caches the bytes you forwarded. Compression changed those bytes, so your original messages and the ones the provider cached are no longer the same thing — and it is the forwarded version you have to keep reproducing. Send the pristine originals again next turn and the provider sees a different prefix and re-reads it from scratch. On Anthropic a cache read is ~90% cheaper than fresh input, so that can easily cost more than the compression saves.

Compression is also not uniform over a conversation: how hard a message is compressed depends partly on how far it now sits from the end, so an older tool result can fall outside the recent-read protection window as the conversation grows and be compressed harder than it was last turn. Another reason not to rely on re-compression reproducing earlier output.

Two rules:

1. Pass config.frozen_message_count — how many leading messages the provider has already cached.
2. Send back your own previous output, not the original messages. frozen_message_count returns those leading messages exactly as you passed them in — it pins whatever you hand it. Hand it pristine originals and you get pristine originals back, which is precisely the prefix the provider does not have.

# Keep what you FORWARDED, not what you started with.
forwarded: list[dict] = []

def next_turn(new_messages: list[dict]) -> list[dict]:
    body = {
        "messages": forwarded + new_messages,
        "model": "claude-sonnet-4-6",
        # Everything already forwarded is already cached upstream — pin it.
        "config": {"frozen_message_count": len(forwarded)},
    }
    result = requests.post(f"{proxy}/v1/compress", json=body).json()
    forwarded[:] = result["messages"]   # becomes next turn's frozen prefix
    return forwarded

Also for multi-turn callers:

• Leave config.mode unset. The default is marker-free, which is what a forward-only caller wants.
• Send the real model name so the tokenizer and context limit resolve correctly — including gateway-prefixed forms.
• protect_recent is not a substitute. It guards the newest messages; frozen_message_count guards the oldest, which is the cached end.

Agent wrapping

Use headroom wrap to launch supported CLI agents through the local proxy:

# Claude Code
headroom wrap claude

# Claude Code extension in VS Code (configures settings, then starts the proxy)
headroom wrap vscode-claude

# OpenAI Codex
headroom wrap codex

# Aider
headroom wrap aider

# Cursor (starts the proxy and prints settings to paste into Cursor)
headroom wrap cursor

# Grok Build (updates ~/.grok/config.toml and starts the proxy)
headroom wrap grok-build

Cursor reads model endpoints from its settings UI, so headroom wrap cursor does not rewrite Cursor configuration or launch the app. After it starts the proxy, copy the printed base URL into Cursor's model settings.

Grok Build reads model endpoints from ~/.grok/config.toml. headroom wrap grok-build injects or updates [model.grok-build] base_url to point at the local proxy, then run grok from the same project directory. See Grok Build Integration.

The official Claude Code extension reads Claude Code's user settings rather than the terminal environment. Use headroom wrap vscode-claude, reload VS Code after the first run, and keep the wrapper running. See the VS Code Claude Code guide for verification and undo steps.

For environment-driven clients, you can also set the base URL manually:

# Claude Code
ANTHROPIC_BASE_URL=http://localhost:8787 claude

# Any OpenAI-compatible CLI client that reads OPENAI_BASE_URL
OPENAI_BASE_URL=http://localhost:8787/v1 your-client

Cloud providers

# AWS Bedrock
headroom proxy --backend bedrock --region us-east-1

# Google Vertex AI
headroom proxy --backend vertex_ai --region us-central1

# Azure OpenAI
headroom proxy --backend azure

# OpenRouter (400+ models)
OPENROUTER_API_KEY=sk-or-... headroom proxy --backend openrouter

Google Vertex AI

--backend vertex_ai delegates to LiteLLM, which brings two requirements that are easy to miss:

1. Install the Vertex SDK. google-cloud-aiplatform is not included in any Headroom extra ([proxy], [all], …) or Docker image variant, so install it alongside Headroom:

pip install "headroom-ai[proxy]" "google-cloud-aiplatform>=1.38"

Without it, the first Vertex request fails with litellm.BadRequestError: … vertexai import failed … No module named 'vertexai'.

2. Set the LiteLLM project/location variables. LiteLLM reads the GCP project and region from VERTEXAI_PROJECT / VERTEXAI_LOCATION — these are not the standard Google Cloud variables (GOOGLE_CLOUD_PROJECT / GOOGLE_CLOUD_LOCATION) used by gcloud and current Google SDKs:

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json  # or use ADC
export VERTEXAI_PROJECT=<your-gcp-project>
export VERTEXAI_LOCATION=us-central1

headroom proxy --backend vertex_ai --region us-central1

Backend name aliases. vertex_ai, vertex, google-vertex, googlevertex, litellm-vertex, and litellm-vertex_ai are all normalized to the same LiteLLM-backed vertex_ai backend — CLI help text and older docs use these spellings interchangeably.

Running Claude Code against Claude models on Vertex? See Claude Code on Vertex AI for the recommended native Vertex-mode flow that reuses Claude Code's own GCP auth.

Native Vertex passthrough routes

Separately from --backend vertex_ai, the proxy always registers routes that mirror Vertex's native REST shape verbatim — no backend flag needed:

/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:generateContent
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:streamGenerateContent
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:countTokens
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:rawPredict
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:streamRawPredict

Requests with publisher=google (Gemini models) go through Headroom's full Gemini optimization handler; publisher=anthropic (Claude on Vertex) routes through the same LiteLLM-Vertex path as --backend vertex_ai. Any client that already speaks the native Vertex REST API can simply point its endpoint at the proxy — this is the mechanism headroom wrap claude uses in Vertex mode.

Bedrock via a local gateway

--backend bedrock accepts Anthropic input (/v1/messages) and re-signs to AWS. Some setups are the other way around: the client already speaks Bedrock (e.g. Claude Code with CLAUDE_CODE_USE_BEDROCK=1, or any AWS SDK pointed at a custom endpoint), sending POST /model/{id}/invoke to a local gateway that re-signs and forwards to AWS (LiteLLM, LocalStack, a corporate Bedrock proxy).

--bedrock-api-url lets Headroom sit in that chain. It registers passthrough routes for /model/{id}/invoke and /model/{id}/invoke-with-response-stream, compresses the request body with the same pipeline as /v1/messages, and forwards to the gateway:

headroom proxy --bedrock-api-url http://127.0.0.1:4000
# then point the client's Bedrock endpoint at Headroom:
AWS_ENDPOINT_URL_BEDROCK_RUNTIME=http://127.0.0.1:8787 your-bedrock-client

The routes are registered only when --bedrock-api-url (or BEDROCK_TARGET_API_URL) is set — otherwise Bedrock requests fall through unchanged.

Environment variables

export HEADROOM_HOST=0.0.0.0
export HEADROOM_PORT=8787
export HEADROOM_BUDGET=100.0

# Route OpenAI passthrough requests to a custom endpoint
export OPENAI_TARGET_API_URL=https://custom.openai.endpoint.com

# Route Anthropic passthrough requests to a custom endpoint
export ANTHROPIC_TARGET_API_URL=https://litellm.company.internal

# Compress Bedrock InvokeModel traffic, forwarding to a re-signing gateway
export BEDROCK_TARGET_API_URL=http://127.0.0.1:4000

headroom proxy

Production deployment

gunicorn

pip install gunicorn

gunicorn headroom.proxy.server:app \
  --workers 4 \
  --bind 0.0.0.0:8787 \
  --worker-class uvicorn.workers.UvicornWorker

Docker

FROM python:3.11-slim
RUN apt-get update && apt-get install -y --no-install-recommends build-essential \
    && pip install "headroom-ai[proxy]" \
    && apt-get purge -y build-essential && apt-get autoremove -y \
    && rm -rf /var/lib/apt/lists/*
EXPOSE 8787
CMD ["headroom", "proxy", "--host", "0.0.0.0"]