# Migrate from LiteLLM in 5 minutes Move your apps from a LiteLLM proxy to Kaeso with a new base URL and key, then carry the rest of your config.yaml across. Web page: https://kaeso.app/articles/migrate-from-litellm In other languages: [Deutsch](https://kaeso.app/de/articles/migrate-from-litellm.md) · [简体中文](https://kaeso.app/zh-CN/articles/migrate-from-litellm.md) By Kaeso Team · 5 November 2026 If your apps call a LiteLLM proxy today, they already speak the OpenAI or the Anthropic API. Kaeso speaks both. So moving an app is mostly a change of address: a new base URL and a new key. Your code, your SDK and the way you stream replies stay as they are. The five minutes are for that switch: starting Kaeso, connecting one provider and pointing one app at it. A long `config.yaml` takes longer to carry across, so the second half of this guide goes through it piece by piece: models, fallbacks, keys, budgets, caching and logs, including what works differently and the few things Kaeso doesn't do. Keep LiteLLM running while you move. Nothing here changes it, so any app can go back by changing its base URL back. ## Before you start - **A server.** Linux with Docker Engine and Docker Compose 2.24 or newer, 2 vCPUs, 4 GB of memory and 20 GB of disk. To try Kaeso on your own computer first, use `localhost` as the domain. - **A domain** whose DNS record points at the server, with ports 80 and 443 open, so the bundled Caddy can get a certificate. - **Your provider keys.** The ones LiteLLM uses today; your `config.yaml` names them as `os.environ/OPENAI_API_KEY` and the like. - **An SMTP server** for sign-in links and invitations. You can leave it out while you try Kaeso. ## The switch ### Start Kaeso Each release publishes a `compose.yaml` that runs the gateway, the console, PostgreSQL, ClickHouse for request logs, Valkey and Caddy. Download it, write the settings and start everything: ```bash mkdir -m 700 kaeso && cd kaeso curl -fsSLO https://github.com/logicplanes/kaeso/releases/latest/download/compose.yaml umask 077 cat > .env < master-keys sudo chown 65532:65532 master-keys && sudo chmod 0400 master-keys docker compose up -d --wait docker compose exec gateway kaeso setup-link ``` The first start takes a minute or two while the images download. On Docker Desktop or Colima, run only `chmod 0400 master-keys` instead of the `chown` line. `KAESO_MAIL_TRANSPORT=log` writes emails to the gateway's log instead of sending them, which is enough to try Kaeso alone. Before you invite anyone, replace it with your SMTP server ([email settings](https://docs.kaeso.app/self-hosting/email)). One name means two different things here. Kaeso's master key encrypts the provider keys it stores; it isn't LiteLLM's `master_key`, and nothing calls models with it. Keep a copy of `master-keys` somewhere safe, apart from your database backups. The last command prints a setup link. Open it, enter your name, your email, a password and a name for your organization, and choose **Set up Kaeso**. You are now the instance admin and the owner of the organization. ### Connect a provider Open **Providers**, choose **Add provider key**, pick the provider and paste its key. The first key of a provider becomes its default, which every project uses. Kaeso stores the key encrypted and never shows it again. Apps never see it: they call Kaeso with keys of their own. ### Add your models Requests can only name models in a project's catalogue, much as LiteLLM serves only what `model_list` names. On **Projects**, choose **Create project** and name it after your app. Then open **Models**, choose **Add models**, search for the models your app calls, such as `claude-sonnet-5-5` or `gpt-6-sol`, and add them. You don't enter prices or context windows. They come from [KPMR](https://kpmr.kaeso.app), the Kaeso Provider & Model Registry, and the install keeps them current by itself. ### Create a key In the project, choose **API keys**, then **Create key**. Name it after the app, keep the access **Inference**, and choose an expiry. Copy the key, `sk-kso-…`, before you close the dialog: Kaeso keeps only a hash of it. ### Point the app at Kaeso Change the base URL and the key. Nothing else: ```diff import os from openai import OpenAI client = OpenAI( - base_url="http://litellm.internal:4000", - api_key=os.environ["LITELLM_API_KEY"], + base_url="https://kaeso.example.com/v1", + api_key=os.environ["KAESO_API_KEY"], ) ``` If the app reads its settings from the environment, change those instead: ```bash OPENAI_BASE_URL=https://kaeso.example.com/v1 OPENAI_API_KEY=sk-kso-… ANTHROPIC_BASE_URL=https://kaeso.example.com ANTHROPIC_API_KEY=sk-kso-… ``` Mind the `/v1`. LiteLLM answers on `/chat/completions` as well as on `/v1/chat/completions`, so many apps point the OpenAI SDK at the bare host. Kaeso serves its model API under `/v1` only, and the rest of the domain is the console. The Anthropic SDK is the other way round: it adds `/v1` itself, so its base URL is the bare domain. | SDK | Base URL | Sends the key as | | --- | --- | --- | | OpenAI | `https://kaeso.example.com/v1` | `Authorization: Bearer` | | Anthropic | `https://kaeso.example.com` | `x-api-key` or `Authorization: Bearer` | Send a request. Every answer carries a `kaeso-request-id` header, and the project's **Request logs** shows the request a few seconds later with its status, tokens, cost and timing. That's the switch. As with LiteLLM, any SDK reaches any chat model, so the OpenAI SDK can call Claude. When the app sends a field that the model's provider can't take, Kaeso doesn't drop it: it refuses the request with an error that names the field, so no setting goes missing without you knowing. ### Claude Code and Codex Coding tools are apps like any other. For Claude Code, change two values in `~/.claude/settings.json`: ```json { "env": { "ANTHROPIC_BASE_URL": "https://kaeso.example.com", "ANTHROPIC_AUTH_TOKEN": "sk-kso-…" } } ``` Claude Code calls an Opus, a Sonnet and a Haiku model, the last one for background tasks, so all three need to be in the project's catalogue. `ANTHROPIC_DEFAULT_OPUS_MODEL`, `ANTHROPIC_DEFAULT_SONNET_MODEL` and `ANTHROPIC_DEFAULT_HAIKU_MODEL` choose which ones, and can name an alias too. For Codex, add Kaeso as a model provider with `base_url = "https://kaeso.example.com/v1"` and `wire_api = "responses"`. [Coding tools](https://docs.kaeso.app/get-started/coding-tools) has both files in full. ## Your config.yaml, line by line Here is a LiteLLM config of the kind many teams run: three models, one of them under a name of its own, fallbacks, retries, a Redis cache and a database for keys and spend. ```yaml model_list: - model_name: claude-sonnet-5-5 litellm_params: model: anthropic/claude-sonnet-5-5 api_key: os.environ/ANTHROPIC_API_KEY - model_name: gpt-6-sol litellm_params: model: openai/gpt-6-sol api_key: os.environ/OPENAI_API_KEY - model_name: fast litellm_params: model: openai/gpt-5.4-mini api_key: os.environ/OPENAI_API_KEY litellm_settings: num_retries: 2 fallbacks: [{"claude-sonnet-5-5": ["gpt-6-sol"]}] context_window_fallbacks: [{"fast": ["claude-sonnet-5-5"]}] cache: true cache_params: {type: redis} general_settings: master_key: os.environ/LITELLM_MASTER_KEY database_url: os.environ/DATABASE_URL ``` In Kaeso, none of this lives in a file. You set it in the console, or from a script with the Platform API, and the audit log records every change. | In config.yaml | In Kaeso | | --- | --- | | `api_key: os.environ/…` | A provider key on **Providers**, stored encrypted | | `model_list` | The project's catalogue on **Models** | | `model_name: claude-sonnet-5-5` | The model itself: apps keep sending the same name | | `model_name: fast` | An alias named `fast` | | `fallbacks`, `context_window_fallbacks` | The alias's route, with fallbacks | | `num_retries` | The SDK's own retries | | `cache: true` | The project's response cache | | `master_key` | Console accounts for people, platform keys for scripts | | `database_url`, Redis | PostgreSQL, ClickHouse and Valkey from `compose.yaml` | | Virtual keys | API keys, `sk-kso-…`, one per app | | Teams | Projects | | `max_budget`, `tpm_limit`, `rpm_limit` | Limits on a project, a member or a key | The sections below take these one at a time. ## Models and providers ### Names your apps already send A `model_name` that is the provider's own model id, such as `claude-sonnet-5-5`, needs nothing more than the model in the catalogue. Apps keep sending it. A name of your own, such as `fast`, becomes an alias. On **Models**, open **Aliases**, choose **Create alias**, name it `fast` and choose `gpt-5.4-mini`. Apps send `fast`, and you can move it to another model later without touching them. Kaeso also reads the `provider/model` form, so a name such as `anthropic/claude-sonnet-5-5` works as it is. An alias can't contain `/`, though, so a LiteLLM name such as `prod/claude` has to change in the app. ### Provider prefixes LiteLLM's prefix in `litellm_params.model` tells you which provider key to add. Most are the same in Kaeso; a few are spelled differently: | LiteLLM | Kaeso | | --- | --- | | `openai/`, `anthropic/`, `groq/`, `mistral/`, `deepseek/`, `xai/`, `cerebras/`, `openrouter/` | The same id | | `together_ai/` | `togetherai` | | `cohere/`, `cohere_chat/` | `cohere` | | `moonshot/` | `moonshotai` | | `dashscope/` | `alibaba`, QwenCloud | | `vercel_ai_gateway/` | `vercel` | Two shortcuts help with large configs. **Add every current OpenAI model**, and the same button for each other provider, stands in for a wildcard such as `openai/*`; it adds the models the provider has today, not ones that appear later. And when two providers serve the same model id, name the one you mean, such as `groq/openai/gpt-oss-120b`. ### Prices you set yourself LiteLLM takes `input_cost_per_token` and `output_cost_per_token` in US dollars per token. Kaeso's prices are in US dollars per million tokens, so multiply by a million: `0.000003` becomes `3`. Open the model's sheet, choose **Edit overrides** and enter the price. It applies to this project only, and **Reset to KPMR** undoes it. ### Self-hosted servers An entry with an `api_base`, such as a vLLM, Ollama or LM Studio server, becomes a custom provider. On **Providers**, open **Custom providers**, choose **Add custom provider**, enter the base URL and key, and choose **Detect settings**. Kaeso asks the server how to call it, then lists its models when you add them to a project. Kaeso can't know what your server costs, so you enter each model's prices; `0` is right for hardware you don't pay per token. If the server sits at a private address, allow its network in `compose.override.yaml`, next to `compose.yaml`, and apply it with `docker compose up -d --wait`: ```yaml services: gateway: environment: KAESO_CUSTOM_PROVIDERS_ALLOWED_NETWORKS: 10.0.0.0/8 ``` ### Providers Kaeso doesn't have Kaeso has fourteen providers built in: Anthropic, Cerebras, Cohere, DeepSeek, Groq, Mistral, Moonshot, Ollama Cloud, OpenAI, OpenRouter, QwenCloud, SpaceX, Together AI and Vercel AI Gateway. Azure OpenAI, Amazon Bedrock, Google Vertex AI and the Gemini API aren't among them. If your config uses `azure/`, `bedrock/`, `vertex_ai/` or `gemini/`, you have two ways forward. Gemini models are also served by OpenRouter and by Vercel AI Gateway, such as `openrouter/google/gemini-2.5-pro`. For the rest, keep LiteLLM behind Kaeso, as the end of this guide shows. ## Fallbacks, retries and load balancing ### Fallbacks In LiteLLM, fallbacks are a list keyed by model name. In Kaeso they belong to an alias: its route tries one model, then the next. Routes come with Kaeso 0.6.0, which is in release candidates as this is written; on 0.5.6 an alias points at a single model. To keep the name your apps send, give the alias that name. An alias wins over a model of the same name, and the list marks it **Shadows a model**. For the config above: 1. Create an alias named `claude-sonnet-5-5`. 2. Under **Start from**, choose **A fallback chain**. 3. Make `claude-sonnet-5-5` the first step and `gpt-6-sol` its fallback, then choose **Save**. A step falls back when its model is unavailable, rate limited, or can't take something in the request, such as images. Those are `unavailable`, `limited` and `unsupported`, the defaults. A fourth reason, `rejected`, covers what `context_window_fallbacks` and `content_policy_fallbacks` did: the provider refused the request itself, for a context that is too long or for its content. Add it only to a fallback that can take what the first model refused. For long prompts there is a better way than waiting for an error. **Start from** also offers **Long prompts elsewhere**: a condition on the request's input tokens, so a long prompt goes straight to the model with the bigger context. Every answer to an alias carries a `kaeso-route` header that lists each step the request tried, with its status. The request log keeps the same steps under **Tried first**, and the **Test** tab shows where a request would go before you save. ### Retries Kaeso sends each request once. The OpenAI and Anthropic SDKs already retry on their own, twice by default, when an error allows it. If you relied on `num_retries` to retry at the proxy, raise the client's `max_retries` (`maxRetries` in TypeScript) instead. ### Load balancing Several entries under one `model_name`, for example the same model with two keys, have no direct equivalent. A project uses one key per provider, and Kaeso doesn't pool keys or pick a deployment by latency or usage. To spread work across provider accounts, assign a provider key to some projects with **Assign to projects**. To divide traffic between models, use a split in a route: it sends each request down one of its branches by weight, per request, per caller or per value of a header. ## Keys, teams and budgets ### New keys for every app LiteLLM's virtual keys don't come along: Kaeso issues its own and keeps only their hashes. Each app gets a new key, which is the one change you make to it anyway. Teams become projects. Each project has its own catalogue, keys, limits and logs, and people join it with a role: Admin, Developer or Viewer. Give software a service account rather than a person's key, so its keys keep working when that person leaves. ### What a virtual key's settings become | `/key/generate` | Kaeso | | --- | --- | | `key_alias` | The key's name | | `models` | The key's **Models** | | `duration` | The expiry: 1, 7, 30 or 90 days, a date, or never | | `max_budget` with `budget_duration` | A spend budget per day, per month or in total | | `tpm_limit` | A token rate | | `rpm_limit` | A request rate | | `team_id` | The project | | `user_id` | The member or service account that owns the key | Kaeso's days and months are calendar days and months in UTC, so a `budget_duration` of `30d` becomes a monthly budget and `1d` a daily one. A budget is hard by default and refuses requests once it is used up; turn that off and it only sends alerts, by default at 50, 80 and 100 percent. Kaeso has no limit on parallel requests, so `max_parallel_requests` has no counterpart, and neither has a key's `metadata`: its name is what request logs and usage show. Limits nest. A key's budget can't be higher than its owner's, and an owner's can't be higher than the project's, so a team budget in LiteLLM becomes a limit on the project. ### Many keys at once With dozens of keys, use the Platform API. Create a platform key under **Organization settings** › **Platform keys**, then create a service account and a key with its limits for each app: ```bash ORG_URL=https://kaeso.example.com/platform/v1/orgs/org_… PROJECT=proj_… ACCOUNT=$(curl -s "$ORG_URL/projects/$PROJECT/service-accounts" \ -H "Authorization: Bearer $KAESO_PLATFORM_KEY" \ -H "content-type: application/json" \ -d '{"name": "support-bot", "role": "developer"}' | jq -r .id) curl -s "$ORG_URL/projects/$PROJECT/service-accounts/$ACCOUNT/keys" \ -H "Authorization: Bearer $KAESO_PLATFORM_KEY" \ -H "content-type: application/json" \ -d '{ "name": "support-bot", "expires_in_days": 90, "limits": [ {"kind": "spend_budget", "period": "month", "amount": "50"}, {"kind": "request_rate", "period": "minute", "amount": 600} ] }' | jq -r .secret ``` The last command prints the new key once. Store it where the app reads it. [Automate with the Platform API](https://docs.kaeso.app/api/automation) covers the rest, from adding models to rotating keys. ## Caching `cache: true` becomes a switch per project. Open the project's **Settings**, turn on **Cache replies** under **Response cache**, and choose how long to keep replies, from 5 minutes to 7 days. The cache answers a request only when it is identical to an earlier one, compared after translation, so an OpenAI app and an Anthropic app can share an entry. It stores non-streamed replies and is shared only within the project. There is no semantic cache. Apps control it per request with the standard `Cache-Control` header: `no-cache` asks the provider again, `no-store` skips the cache. Every answer says what happened in `kaeso-cache`, and a hit costs nothing against budgets. ## Logs and spend Kaeso keeps its own record of every request: the model, the provider, the status, tokens, cost, timing, the key and who sent it, and by default what was sent and returned. The organization chooses how long: 90 days of metadata and 30 days of content unless you change it. To keep no prompts at all, as `turn_off_message_logging` did, turn off **Log request content** under **Organization settings** › **Logging**, or turn it off for a single project. Usage reads by day, model, key or member, in the console or from the Platform API, and the request logs export as CSV or JSON Lines. The gateway serves Prometheus metrics on its own port for your dashboards. Kaeso doesn't forward its logs to outside tools, so callbacks such as Langfuse or Datadog have no counterpart; read the Platform API's logs or the export instead. Spend history doesn't move either. Kaeso counts from the first request it serves, so keep a dump of LiteLLM's database if you need the history. ## Keep LiteLLM for what Kaeso doesn't do Kaeso's model API covers Chat Completions, Responses, Messages, Embeddings, token counts and the list of models. It has no endpoints for images, audio, rerank, moderations, files or batches, no pass-through routes and no guardrails, and it doesn't proxy MCP servers for your apps. It also refuses Responses requests that ask the provider to store or continue a response, such as `previous_response_id`; send the whole conversation each time. For those endpoints, keep calling LiteLLM directly; nothing stops an app from using both. For a provider Kaeso doesn't have, you can go a step further and put LiteLLM behind Kaeso as a custom provider: 1. In LiteLLM, create a virtual key limited to the models Kaeso should reach, such as your Bedrock and Azure entries. 2. If LiteLLM sits at a private address, allow its network in `compose.override.yaml`, as for a self-hosted server. 3. In Kaeso, choose **Add custom provider** with LiteLLM's address, such as `http://10.0.0.12:4000/v1`, and that key. **Detect settings** finds Chat Completions. Set the **Id** to `litellm`. 4. Add its models to a project and enter their prices. Kaeso charges them at those prices and holds budgets to them. Apps then call LiteLLM's `model_name` with the id in front, such as `litellm/bedrock-claude`, or an alias you name after it, through the same key and base URL as every other model. Kaeso's keys, limits and logs apply to them as to any other request. ## Switching back, and switching off Until you stop LiteLLM, rolling back is the same change in reverse: the old base URL and the old key. Move one app at a time, and watch its requests appear in Kaeso's request logs. Once every app's traffic shows up in Kaeso and LiteLLM's logs go quiet, take a last dump of its database and stop it. If a part of your setup didn't fit any of the sections above, [tell us](https://kaeso.app/contact). It helps decide what Kaeso does next.