Metrics & Monitoring

Headroom provides comprehensive metrics for monitoring compression performance, cost savings, and system health through both the proxy server and the SDK.

Proxy Endpoints

Stats Endpoint

curl http://localhost:8787/stats

{
  "persistent_savings": {
    "lifetime": {
      "tokens_saved": 12500,
      "compression_savings_usd": 0.04
    }
  },
  "requests": {
    "total": 42,
    "cached": 5,
    "rate_limited": 0,
    "failed": 0
  },
  "tokens": {
    "input": 50000,
    "output": 8000,
    "saved": 12500,
    "savings_percent": 25.0
  },
  "cost": {
    "total_cost_usd": 0.15,
    "total_savings_usd": 0.04
  },
  "cache": {
    "entries": 10,
    "total_hits": 5
  }
}

Persistent savings are stored at ~/.headroom/proxy_savings.json and survive proxy restarts. Override the path with HEADROOM_SAVINGS_PATH.

Historical Savings

curl http://localhost:8787/stats-history

Returns durable compression history with hourly, daily, weekly, and monthly rollups. Supports CSV export:

curl "http://localhost:8787/stats-history?format=csv&series=daily"
curl "http://localhost:8787/stats-history?format=csv&series=monthly"

Prometheus Metrics

curl http://localhost:8787/metrics

# HELP headroom_requests_total Total requests processed
headroom_requests_total{mode="optimize"} 1234

# HELP headroom_tokens_saved_total Total tokens saved
headroom_tokens_saved_total 5678900

# HELP headroom_persistent_savings_tokens_saved_total Durable lifetime input tokens saved by proxy compression
headroom_persistent_savings_tokens_saved_total 5678900

# HELP headroom_compression_ratio Compression ratio histogram
headroom_compression_ratio_bucket{le="0.5"} 890
headroom_compression_ratio_bucket{le="0.7"} 1100
headroom_compression_ratio_bucket{le="0.9"} 1200

# HELP headroom_latency_seconds Request latency histogram
headroom_latency_seconds_bucket{le="0.01"} 800
headroom_latency_seconds_bucket{le="0.1"} 1150

# HELP headroom_cache_hits_total Cache hit counter
headroom_cache_hits_total 456

OpenTelemetry (OTLP) Export

The proxy can also push its counters to any OTLP/HTTP endpoint. Install the extra and set four variables:

pip install "headroom-ai[proxy,otel]"

HEADROOM_OTEL_METRICS_ENABLED=1
HEADROOM_OTEL_METRICS_ENDPOINT=http://127.0.0.1:4318/v1/metrics
HEADROOM_OTEL_SERVICE_NAME=headroom-proxy
HEADROOM_OTEL_RESOURCE_ATTRIBUTES=deployment.environment=prod

Exported counters include headroom.proxy.requests, headroom.proxy.tokens.input, and headroom.proxy.tokens.output. headroom.proxy.tokens.saved is the all-layer total: message/compression savings plus tool-schema deferral savings. The component counter headroom.proxy.tokens.tool_schema_saved exposes the deferral portion separately; headroom.compression.tokens.saved remains the compression-pipeline component.

Confirm the exporter is live with curl -s http://localhost:8787/stats | jq .otel.

If your application already configures a global OTEL meter provider, leave HEADROOM_OTEL_* unset — Headroom records into the ambient provider automatically.

Dynatrace

Point the exporter at your environment's OTLP API and add the API token as a header. The token needs the metrics.ingest scope.

HEADROOM_OTEL_METRICS_ENABLED=1
HEADROOM_OTEL_METRICS_ENDPOINT="https://<env-id>.live.dynatrace.com/api/v2/otlp/v1/metrics"
HEADROOM_OTEL_METRICS_HEADERS="Authorization=Api-Token dt0c01.XXXX"
OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=DELTA
HEADROOM_OTEL_SERVICE_NAME=headroom-proxy

Restart the proxy, then search the Dynatrace metric explorer for headroom.proxy.tokens.saved — data appears within ~30s.

For an ActiveGate deployment, swap the base URL for https://<activegate>:9999/e/<env-id>/api/v2/otlp/v1/metrics. If you already run an OpenTelemetry Collector, send Headroom to it instead and add the cumulativetodelta processor — then the temporality variable is unnecessary and the collector holds the token.

Trace export is separate: Headroom's self-configured tracing targets Langfuse only. To land its spans in Dynatrace, leave HEADROOM_LANGFUSE_* unset and run the proxy under opentelemetry-instrument with the standard OTEL_EXPORTER_OTLP_ENDPOINT / OTEL_EXPORTER_OTLP_HEADERS / OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf variables; Headroom records into the ambient tracer provider.

Health Check

curl http://localhost:8787/health

{
  "status": "healthy",
  "version": "0.1.0",
  "uptime_seconds": 3600
}

SDK Metrics

import { HeadroomClient } from 'headroom-ai';

const client = new HeadroomClient();

// Get proxy stats
const stats = await client.proxyStats();
console.log(`Tokens saved: ${stats.tokens.saved}`);
console.log(`Savings: ${stats.tokens.savings_percent}%`);

import { compress } from 'headroom-ai';

const result = await compress(messages, { model: 'gpt-4o' });
console.log(`Tokens: ${result.tokensBefore} -> ${result.tokensAfter}`);
console.log(`Saved: ${result.tokensSaved} (${(result.compressionRatio * 100).toFixed(1)}%)`);
console.log(`Transforms: ${result.transformsApplied.join(', ')}`);

stats = client.get_stats()
print(f"Mode: {stats['config']['mode']}")
print(f"Tokens saved: {stats['session']['tokens_saved_total']}")
print(f"Avg compression: {stats['session']['compression_ratio_avg']:.1%}")

{
    "session": {
        "requests_total": 10,
        "tokens_input_before": 50000,
        "tokens_input_after": 35000,
        "tokens_saved_total": 15000,
        "tokens_output_total": 8000,
        "cache_hits": 3,
        "compression_ratio_avg": 0.70,
    },
    "config": {
        "mode": "optimize",
        "provider": "openai",
        "cache_optimizer_enabled": True,
        "semantic_cache_enabled": False,
    },
    "transforms": {
        "smart_crusher_enabled": True,
        "cache_aligner_enabled": True,
        "rolling_window_enabled": True,
    },
}

from datetime import datetime, timedelta

metrics = client.get_metrics(
    start_time=datetime.utcnow() - timedelta(hours=1),
    limit=100,
)

for m in metrics:
    print(f"{m.timestamp}: {m.tokens_input_before} -> {m.tokens_input_after}")

summary = client.get_summary()
print(f"Total requests: {summary['total_requests']}")
print(f"Total tokens saved: {summary['total_tokens_saved']}")
print(f"Average compression: {summary['avg_compression_ratio']:.1%}")
print(f"Total cost savings: ${summary['total_cost_saved_usd']:.2f}")

Logging

import logging

# INFO level shows compression summaries
logging.basicConfig(level=logging.INFO)

# DEBUG level shows detailed transform decisions
logging.basicConfig(level=logging.DEBUG)

INFO:headroom.transforms.pipeline:Pipeline complete: 45000 -> 4500 tokens (saved 40500, 90.0% reduction)
INFO:headroom.transforms.smart_crusher:SmartCrusher applied top_n strategy: kept 15 of 1000 items
DEBUG:headroom.transforms.smart_crusher:Kept items: [0,1,2,42,77,97,98,99] (errors at 42, warnings at 77)

# Log to file
headroom proxy --log-file headroom.jsonl

# Increase verbosity
headroom proxy --log-level debug

Cost Tracking

Budget Alerts

Set a budget limit in the proxy:

headroom proxy --budget 10.00

When the budget is exceeded, requests return a budget exceeded error, the /stats endpoint shows budget status, and logs indicate the budget state.

Measured vs Estimated Spend

Every cost record carries a basis — where its input-token count came from. When a provider response includes a usage breakdown, the basis is measured. When it doesn't, Headroom substitutes its own tokens_sent count so input cost isn't dropped from the budget, and the record's basis is estimated. Headroom logs one warning per model the first time this happens.

/stats keeps the two separable under cost.budget_basis:

{
  "cost": {
    "budget_limit_usd": 10.0,
    "budget_period": "daily",
    "budget_estimated_basis": "count",
    "budget_basis": {
      "total_usd": 3.1400,
      "measured_usd": 2.9000,
      "estimated_usd": 0.2400,
      "estimated_pct": 7.6,
      "records": 412,
      "estimated_records": 31
    }
  }
}

An estimate can drift in either direction, so you choose what it does to the hard limit:

headroom proxy --budget 10.00 --budget-estimated-basis count   # default

Env: HEADROOM_BUDGET_ESTIMATED_BASIS. headroom doctor reports the estimated share alongside the budget check.

Key Metrics to Monitor

Grafana Dashboard

A ready-to-import dashboard ships in examples/grafana/headroom-dashboard.json. Import it in Grafana (Dashboards → New → Import → Upload) and pick your Prometheus datasource — it renders tokens saved, request throughput, and processing overhead (headroom_overhead_ms_sum / headroom_overhead_ms_count) straight from the metrics the /metrics endpoint exposes.

Example ad-hoc queries against the same metric family: