How Compression Works

Headroom automatically detects what kind of content you're sending and routes it to the right compressor. You don't need to configure anything -- just call compress() and the pipeline handles the rest.

The Pipeline

Every request flows through a short pipeline:

┌──────────────┐     ┌────────────────┐
│ CacheAligner │────>│ ContentRouter  │
│ (off by      │     │                │
│  default)    │     │ Detect type &  │
│ report drift │     │ route to best  │
│ for cache    │     │ compressor     │
└──────────────┘     └────────────────┘

1. CacheAligner (detector-only, off by default) reports dynamic-prefix drift (dates, session context) so callers can keep the static prefix cacheable. It never rewrites your messages.
2. ContentRouter inspects each content block and routes it to one compressor -- SmartCrusher for JSON arrays, CodeAwareCompressor for source code, LogCompressor for build output, and so on. This is where essentially all compression happens.

Content Type Detection

The router auto-detects content type by analyzing structure and patterns. No manual hints required.

Quick Start

import { compress } from "headroom-ai";

const messages = [
  { role: "system" as const, content: "You are a helpful assistant." },
  { role: "user" as const, content: "Summarize this data" },
  { role: "tool" as const, content: '{"results": [...]}', tool_call_id: "call_1" },
];

const result = await compress(messages);
console.log(`Tokens saved: ${result.tokensSaved}`);
console.log(`Compression ratio: ${result.compressionRatio}`);

from headroom import compress

result = compress(content)
print(result.compressed)
print(f"Saved {result.savings_percentage:.0f}% tokens")

Configuring the Compressor

import { compress } from "headroom-ai";

const result = await compress(messages, {
  model: "gpt-4o",
  tokenBudget: 50000,
});

console.log(`Before: ${result.tokensBefore} tokens`);
console.log(`After: ${result.tokensAfter} tokens`);
console.log(`Transforms: ${result.transformsApplied.join(", ")}`);

# Advanced / Internal API -- prefer `from headroom import compress` for typical use
from headroom.compression import UniversalCompressor, UniversalCompressorConfig

config = UniversalCompressorConfig(
    compression_ratio_target=0.5,  # Keep 50% of content
    use_entropy_preservation=True,  # Preserve UUIDs, hashes
    use_magika=True,                # ML-based content detection
    ccr_enabled=True,               # Store originals for retrieval
)

compressor = UniversalCompressor(config=config)
result = compressor.compress(content)

print(f"Type: {result.content_type}")
print(f"Handler: {result.handler_used}")
print(f"Saved: {result.savings_percentage:.0f}%")

Structure Preservation

Headroom doesn't blindly truncate. It identifies what matters in each content type and preserves it:

Real Compression Ratios

Batch Compression

For multiple contents, batch compression is more efficient:

# Advanced / Internal API -- prefer `from headroom import compress` for typical use
from headroom.compression import UniversalCompressor

compressor = UniversalCompressor()

contents = [
    '{"users": [...]}',
    'def hello(): pass',
    'Plain text content',
]

results = compressor.compress_batch(contents)

for result in results:
    print(f"{result.content_type}: {result.savings_percentage:.0f}% saved")

What Happens Under the Hood

When you call compress(), here is the full sequence:

1. Content detection -- Magika (ML-based) or pattern matching identifies the content type
2. Structure extraction -- A handler extracts a structure mask marking what to preserve
3. Compression -- Non-structural content is compressed (SmartCrusher, Kompress ML, or text utilities like TextCrusher)
4. CCR storage -- If enabled, the original is stored for retrieval when the LLM needs full context