If your apps call a LiteLLM proxy today, they already speak the OpenAI or the Anthropic API. Kaeso speaks both. So moving an app is mostly a change of address: a new base URL and a new key. Your code, your SDK and the way you stream replies stay as they are.
The five minutes are for that switch: starting Kaeso, connecting one provider and pointing one app at it. A long config.yaml takes longer to carry across, so the second half of this guide goes through it piece by piece: models, fallbacks, keys, budgets, caching and logs, including what works differently and the few things Kaeso doesn't do.
Keep LiteLLM running while you move. Nothing here changes it, so any app can go back by changing its base URL back.
Before you start
- A server. Linux with Docker Engine and Docker Compose 2.24 or newer, 2 vCPUs, 4 GB of memory and 20 GB of disk. To try Kaeso on your own computer first, use
localhostas the domain. - A domain whose DNS record points at the server, with ports 80 and 443 open, so the bundled Caddy can get a certificate.
- Your provider keys. The ones LiteLLM uses today; your
config.yamlnames them asos.environ/OPENAI_API_KEYand the like. - An SMTP server for sign-in links and invitations. You can leave it out while you try Kaeso.
The switch
Start Kaeso
Each release publishes a compose.yaml that runs the gateway, the console, PostgreSQL, ClickHouse for request logs, Valkey and Caddy. Download it, write the settings and start everything:
The first start takes a minute or two while the images download. On Docker Desktop or Colima, run only chmod 0400 master-keys instead of the chown line.
KAESO_MAIL_TRANSPORT=log writes emails to the gateway's log instead of sending them, which is enough to try Kaeso alone. Before you invite anyone, replace it with your SMTP server (email settings (opens in a new tab)).
One name means two different things here. Kaeso's master key encrypts the provider keys it stores; it isn't LiteLLM's master_key, and nothing calls models with it. Keep a copy of master-keys somewhere safe, apart from your database backups.
The last command prints a setup link. Open it, enter your name, your email, a password and a name for your organization, and choose Set up Kaeso. You are now the instance admin and the owner of the organization.
Connect a provider
Open Providers, choose Add provider key, pick the provider and paste its key. The first key of a provider becomes its default, which every project uses.
Kaeso stores the key encrypted and never shows it again. Apps never see it: they call Kaeso with keys of their own.
Add your models
Requests can only name models in a project's catalogue, much as LiteLLM serves only what model_list names. On Projects, choose Create project and name it after your app. Then open Models, choose Add models, search for the models your app calls, such as claude-sonnet-5-5 or gpt-6-sol, and add them.
You don't enter prices or context windows. They come from KPMR (opens in a new tab), the Kaeso Provider & Model Registry, and the install keeps them current by itself.
Create a key
In the project, choose API keys, then Create key. Name it after the app, keep the access Inference, and choose an expiry. Copy the key, sk-kso-…, before you close the dialog: Kaeso keeps only a hash of it.
Point the app at Kaeso
Change the base URL and the key. Nothing else:
If the app reads its settings from the environment, change those instead:
Mind the /v1. LiteLLM answers on /chat/completions as well as on /v1/chat/completions, so many apps point the OpenAI SDK at the bare host. Kaeso serves its model API under /v1 only, and the rest of the domain is the console. The Anthropic SDK is the other way round: it adds /v1 itself, so its base URL is the bare domain.
Send a request. Every answer carries a kaeso-request-id header, and the project's Request logs shows the request a few seconds later with its status, tokens, cost and timing. That's the switch.
As with LiteLLM, any SDK reaches any chat model, so the OpenAI SDK can call Claude. When the app sends a field that the model's provider can't take, Kaeso doesn't drop it: it refuses the request with an error that names the field, so no setting goes missing without you knowing.
Claude Code and Codex
Coding tools are apps like any other. For Claude Code, change two values in ~/.claude/settings.json:
Claude Code calls an Opus, a Sonnet and a Haiku model, the last one for background tasks, so all three need to be in the project's catalogue. ANTHROPIC_DEFAULT_OPUS_MODEL, ANTHROPIC_DEFAULT_SONNET_MODEL and ANTHROPIC_DEFAULT_HAIKU_MODEL choose which ones, and can name an alias too. For Codex, add Kaeso as a model provider with base_url = "https://kaeso.example.com/v1" and wire_api = "responses". Coding tools (opens in a new tab) has both files in full.
Your config.yaml, line by line
Here is a LiteLLM config of the kind many teams run: three models, one of them under a name of its own, fallbacks, retries, a Redis cache and a database for keys and spend.
In Kaeso, none of this lives in a file. You set it in the console, or from a script with the Platform API, and the audit log records every change.
The sections below take these one at a time.
Models and providers
Names your apps already send
A model_name that is the provider's own model id, such as claude-sonnet-5-5, needs nothing more than the model in the catalogue. Apps keep sending it.
A name of your own, such as fast, becomes an alias. On Models, open Aliases, choose Create alias, name it fast and choose gpt-5.4-mini. Apps send fast, and you can move it to another model later without touching them.
Kaeso also reads the provider/model form, so a name such as anthropic/claude-sonnet-5-5 works as it is. An alias can't contain /, though, so a LiteLLM name such as prod/claude has to change in the app.
Provider prefixes
LiteLLM's prefix in litellm_params.model tells you which provider key to add. Most are the same in Kaeso; a few are spelled differently:
Two shortcuts help with large configs. Add every current OpenAI model, and the same button for each other provider, stands in for a wildcard such as openai/*; it adds the models the provider has today, not ones that appear later. And when two providers serve the same model id, name the one you mean, such as groq/openai/gpt-oss-120b.
Prices you set yourself
LiteLLM takes input_cost_per_token and output_cost_per_token in US dollars per token. Kaeso's prices are in US dollars per million tokens, so multiply by a million: 0.000003 becomes 3. Open the model's sheet, choose Edit overrides and enter the price. It applies to this project only, and Reset to KPMR undoes it.
Self-hosted servers
An entry with an api_base, such as a vLLM, Ollama or LM Studio server, becomes a custom provider. On Providers, open Custom providers, choose Add custom provider, enter the base URL and key, and choose Detect settings. Kaeso asks the server how to call it, then lists its models when you add them to a project.
Kaeso can't know what your server costs, so you enter each model's prices; 0 is right for hardware you don't pay per token. If the server sits at a private address, allow its network in compose.override.yaml, next to compose.yaml, and apply it with docker compose up -d --wait:
Providers Kaeso doesn't have
Kaeso has fourteen providers built in: Anthropic, Cerebras, Cohere, DeepSeek, Groq, Mistral, Moonshot, Ollama Cloud, OpenAI, OpenRouter, QwenCloud, SpaceX, Together AI and Vercel AI Gateway. Azure OpenAI, Amazon Bedrock, Google Vertex AI and the Gemini API aren't among them. If your config uses azure/, bedrock/, vertex_ai/ or gemini/, you have two ways forward. Gemini models are also served by OpenRouter and by Vercel AI Gateway, such as openrouter/google/gemini-2.5-pro. For the rest, keep LiteLLM behind Kaeso, as the end of this guide shows.
Fallbacks, retries and load balancing
Fallbacks
In LiteLLM, fallbacks are a list keyed by model name. In Kaeso they belong to an alias: its route tries one model, then the next. Routes come with Kaeso 0.6.0, which is in release candidates as this is written; on 0.5.6 an alias points at a single model.
To keep the name your apps send, give the alias that name. An alias wins over a model of the same name, and the list marks it Shadows a model. For the config above:
- Create an alias named
claude-sonnet-5-5. - Under Start from, choose A fallback chain.
- Make
claude-sonnet-5-5the first step andgpt-6-solits fallback, then choose Save.
A step falls back when its model is unavailable, rate limited, or can't take something in the request, such as images. Those are unavailable, limited and unsupported, the defaults. A fourth reason, rejected, covers what context_window_fallbacks and content_policy_fallbacks did: the provider refused the request itself, for a context that is too long or for its content. Add it only to a fallback that can take what the first model refused.
For long prompts there is a better way than waiting for an error. Start from also offers Long prompts elsewhere: a condition on the request's input tokens, so a long prompt goes straight to the model with the bigger context.
Every answer to an alias carries a kaeso-route header that lists each step the request tried, with its status. The request log keeps the same steps under Tried first, and the Test tab shows where a request would go before you save.
Retries
Kaeso sends each request once. The OpenAI and Anthropic SDKs already retry on their own, twice by default, when an error allows it. If you relied on num_retries to retry at the proxy, raise the client's max_retries (maxRetries in TypeScript) instead.
Load balancing
Several entries under one model_name, for example the same model with two keys, have no direct equivalent. A project uses one key per provider, and Kaeso doesn't pool keys or pick a deployment by latency or usage. To spread work across provider accounts, assign a provider key to some projects with Assign to projects. To divide traffic between models, use a split in a route: it sends each request down one of its branches by weight, per request, per caller or per value of a header.
Keys, teams and budgets
New keys for every app
LiteLLM's virtual keys don't come along: Kaeso issues its own and keeps only their hashes. Each app gets a new key, which is the one change you make to it anyway.
Teams become projects. Each project has its own catalogue, keys, limits and logs, and people join it with a role: Admin, Developer or Viewer. Give software a service account rather than a person's key, so its keys keep working when that person leaves.
What a virtual key's settings become
Kaeso's days and months are calendar days and months in UTC, so a budget_duration of 30d becomes a monthly budget and 1d a daily one. A budget is hard by default and refuses requests once it is used up; turn that off and it only sends alerts, by default at 50, 80 and 100 percent. Kaeso has no limit on parallel requests, so max_parallel_requests has no counterpart, and neither has a key's metadata: its name is what request logs and usage show.
Limits nest. A key's budget can't be higher than its owner's, and an owner's can't be higher than the project's, so a team budget in LiteLLM becomes a limit on the project.
Many keys at once
With dozens of keys, use the Platform API. Create a platform key under Organization settings › Platform keys, then create a service account and a key with its limits for each app:
The last command prints the new key once. Store it where the app reads it. Automate with the Platform API (opens in a new tab) covers the rest, from adding models to rotating keys.
Caching
cache: true becomes a switch per project. Open the project's Settings, turn on Cache replies under Response cache, and choose how long to keep replies, from 5 minutes to 7 days.
The cache answers a request only when it is identical to an earlier one, compared after translation, so an OpenAI app and an Anthropic app can share an entry. It stores non-streamed replies and is shared only within the project. There is no semantic cache. Apps control it per request with the standard Cache-Control header: no-cache asks the provider again, no-store skips the cache. Every answer says what happened in kaeso-cache, and a hit costs nothing against budgets.
Logs and spend
Kaeso keeps its own record of every request: the model, the provider, the status, tokens, cost, timing, the key and who sent it, and by default what was sent and returned. The organization chooses how long: 90 days of metadata and 30 days of content unless you change it. To keep no prompts at all, as turn_off_message_logging did, turn off Log request content under Organization settings › Logging, or turn it off for a single project.
Usage reads by day, model, key or member, in the console or from the Platform API, and the request logs export as CSV or JSON Lines. The gateway serves Prometheus metrics on its own port for your dashboards.
Kaeso doesn't forward its logs to outside tools, so callbacks such as Langfuse or Datadog have no counterpart; read the Platform API's logs or the export instead. Spend history doesn't move either. Kaeso counts from the first request it serves, so keep a dump of LiteLLM's database if you need the history.
Keep LiteLLM for what Kaeso doesn't do
Kaeso's model API covers Chat Completions, Responses, Messages, Embeddings, token counts and the list of models. It has no endpoints for images, audio, rerank, moderations, files or batches, no pass-through routes and no guardrails, and it doesn't proxy MCP servers for your apps. It also refuses Responses requests that ask the provider to store or continue a response, such as previous_response_id; send the whole conversation each time.
For those endpoints, keep calling LiteLLM directly; nothing stops an app from using both. For a provider Kaeso doesn't have, you can go a step further and put LiteLLM behind Kaeso as a custom provider:
- In LiteLLM, create a virtual key limited to the models Kaeso should reach, such as your Bedrock and Azure entries.
- If LiteLLM sits at a private address, allow its network in
compose.override.yaml, as for a self-hosted server. - In Kaeso, choose Add custom provider with LiteLLM's address, such as
http://10.0.0.12:4000/v1, and that key. Detect settings finds Chat Completions. Set the Id tolitellm. - Add its models to a project and enter their prices. Kaeso charges them at those prices and holds budgets to them.
Apps then call LiteLLM's model_name with the id in front, such as litellm/bedrock-claude, or an alias you name after it, through the same key and base URL as every other model. Kaeso's keys, limits and logs apply to them as to any other request.
Switching back, and switching off
Until you stop LiteLLM, rolling back is the same change in reverse: the old base URL and the old key. Move one app at a time, and watch its requests appear in Kaeso's request logs.
Once every app's traffic shows up in Kaeso and LiteLLM's logs go quiet, take a last dump of its database and stop it. If a part of your setup didn't fit any of the sections above, tell us (opens in a new tab). It helps decide what Kaeso does next.