Memory01
LiteLLM Rust never went above 86 MB and needs no database. Kaeso reached 164 MB, and its PostgreSQL and Valkey took another 271 MB.

Kaeso 0.7.0 against LiteLLM, Bifrost and Portkey on BerriAI’s ai-gateway-bench, the benchmark from the company behind LiteLLM. One machine, one mock provider, the benchmark’s own drivers.
BerriAI/ai-gateway-bench at 44365ca, 22 November 2026 (opens in a new tab)Each gateway got the same two dedicated cores, 64 clients and a mock provider that answers at once, so the gateway itself was the limit.
Requests a second, the median of 3 rounds; the thin line runs from the slowest round to the fastest. The same driver reached 27,750 a second straight to the mock, far above any gateway.
How much longer a request takes through the gateway than straight to the mock, which answers in about 22 ms as a fast provider would.
Added latency in ms, p50 and p99, the median of 3 rounds, with the p99’s range under it. Each run was paired with a direct run of the same shape just before it.
No row for 64 clients: there the mock and the drivers share two cores and top out near 2,900 requests a second, so subtracting the direct run would understate every gateway’s time. The capacity figure measures that load instead.
200 streamed requests at 16 clients, and two coding agents’ recorded sessions replayed turn by turn.
Straight to the mock, the gap between chunks is 7.5 ms at p99. A session is 30 recorded turns sent one after another; added is its whole time through the gateway minus straight to the mock. Every streamed session gave valid tool calls through every gateway.
Bifrost sends its first event before the provider answers, and the benchmark counts that as the first token, so its time to first token reads as negative.
Results Kaeso doesn’t win, and what its extra work costs.
LiteLLM Rust never went above 86 MB and needs no database. Kaeso reached 164 MB, and its PostgreSQL and Valkey took another 271 MB.
With one request at a time, LiteLLM Rust’s p99 was +0.98 ms against Kaeso’s +1.66 ms, and no round of Kaeso’s came closer.
Time to first token (+0.87 against +0.96 ms) and the Claude Code session (+41.5 against +41.0 ms) fall within each other’s range.
With every request logged to ClickHouse, Kaeso served 5,324 requests a second, 16% fewer, and its databases took 1,163 MB.
Every gateway ran as its project ships it, against the mock in Anthropic’s Messages format. Kaeso checked the most on every request.
Everything needed to check a number: the machine, the benchmark, every version and how each run was paired.
ghcr.io/logicplanes/kaeso-gateway:0.7.0sha256:3d32fd7827417e49475b4ac9ad7d25f7daed28f02fa489ef4077ff501faaae1agithub.com/BerriAI/litellm@d4e8f96, litellm-gateway built with rust:1.98.0sha256:b80e284d93b67806511e0371456d80083d6f1d932abd723a1b31c1ed578ae122maximhq/bifrost:v2.6.0sha256:5968d007ba4f74ee01bc4ceb071decb9927a0b7bb43dbb9072208f75e2d522e3portkeyai/gateway:1.15.2sha256:97f094d9c8a764cbfaa2a7138c0017b247ca923bb06db1b4c13b7f8a33b5200dghcr.io/berriai/litellm:v1.110.2sha256:ef798594ca3db8d81bad2e1439f536f49b71fe095ee9f20416f5e149b4e22035Kaeso’s databases: postgres:18.6-alpine, valkey/valkey:9.1.3-alpine and clickhouse/clickhouse-server:26.3.62.3-alpine.
On a fresh Linux machine with four cores, as root. The setup installs Docker, builds the benchmark and LiteLLM Rust at the same commits and pulls every image by its digest.
curl -fsSLO https://kaeso.app/benchmarks/2026-11-22/harness.tar.gztar -xzf harness.tar.gzbash kaeso-gateway-bench/setup.shcd /rootai-gateway-bench/.venv/bin/python run.pyai-gateway-bench/.venv/bin/python run_capacity.pyThree rounds of every gateway take about 90 minutes, the capacity pass another 30. Absolute numbers change from machine to machine, so compare gateways measured in the same run.

Install Kaeso next to the gateway you run today and point the same benchmark at both.