LiteLLM

Headroom integrates with LiteLLM as a callback that compresses messages before they reach any provider. One line to enable, works with all 100+ LiteLLM-supported providers.

Installation

pip install headroom-ai litellm

Quick start

import litellm
from headroom.integrations.litellm_callback import HeadroomCallback

litellm.callbacks = [HeadroomCallback()]

# All calls now compressed automatically
response = litellm.completion(model="gpt-4o", messages=[...])
response = litellm.completion(model="bedrock/claude-sonnet", messages=[...])
response = litellm.completion(model="azure/gpt-4o", messages=[...])

The callback compresses messages in LiteLLM's pre_call_hook before they reach the provider.

How it works

1. You call litellm.completion() with your messages
2. HeadroomCallback.pre_call_hook compresses the messages
3. LiteLLM sends the compressed messages to the provider
4. The response comes back unchanged

This works with every provider LiteLLM supports: OpenAI, Anthropic, Bedrock, Azure, Vertex AI, Cohere, Groq, Mistral, Together, Ollama, and more.

With LiteLLM Proxy

If you run LiteLLM as a proxy server, use the ASGI middleware:

from litellm.proxy.proxy_server import app
from headroom.integrations.asgi import CompressionMiddleware

app.add_middleware(CompressionMiddleware)

Or configure via YAML:

# litellm_config.yaml
litellm_settings:
  callbacks: ["headroom.integrations.litellm_callback.HeadroomCallback"]

Direct compress() with LiteLLM

You can also use compress() directly instead of the callback:

import litellm
from headroom import compress

messages = [{"role": "user", "content": large_content}]
compressed = compress(messages, model="bedrock/claude-sonnet")

response = litellm.completion(
    model="bedrock/claude-sonnet",
    messages=compressed.messages,
)

print(f"Saved {compressed.tokens_saved} tokens")

ASGI middleware

Drop-in middleware for any ASGI application. Intercepts /v1/messages, /v1/chat/completions, /v1/responses, and /chat/completions:

from fastapi import FastAPI
from headroom.integrations.asgi import CompressionMiddleware

app = FastAPI()
app.add_middleware(CompressionMiddleware)

Response headers include x-headroom-compressed: true and x-headroom-tokens-saved: 1234.

Over HTTP (guardrail / gateway)

The options above run Headroom in the LiteLLM process. If instead LiteLLM runs as its own proxy and you want it to call Headroom over the network — the guardrail deployment — point it at POST /v1/compress. LiteLLM swaps messages for the compressed result and forwards to the provider.

Two things this deployment needs:

# Headroom is loopback-only by default and answers remote callers with 404.
HEADROOM_COMPRESS_ALLOW_REMOTE=1 headroom proxy

Leave config.mode unset. The default pipeline is marker-free, which is what a forward-only caller wants: mode: "ccr" emits retrieval markers that are a dangling pointer unless you also inject the headroom_retrieve tool and can reach /v1/retrieve.

Because LiteLLM passes model names through, send the real one — claude-sonnet-4-6, bedrock/anthropic.claude-3-5-sonnet, gemini-2.5-pro — so Headroom resolves the right tokenizer and context limit. Anthropic-shaped messages need no conversion; see Message format.

For multi-turn agent loops, set config.frozen_message_count to the number of messages the provider has already cached, and send back the messages you previously forwarded rather than the pristine originals. Getting this wrong silently destroys the provider's prefix cache — see Multi-turn usage for the loop.