Skip to content

Fallbacks, conditions and splits with alias routes

In Kaeso 0.6.0 an alias holds a route. Fall back to another model when one fails, send enterprise traffic or long prompts somewhere else, and A/B test two models by caller, all without changing a line in your app.

Published
Reading time
4 min
Author
Kaeso Team
A bright gradient of orange and yellow, folding diagonally downward, with a warm light in one corner

An app names a model in every request, and that name tends to stay in the code long after the model behind it should have changed. Kaeso's aliases have always helped there: the app sends support, and the project decides which model answers.

In 0.6.0 an alias holds a whole route. It can fall back to another model when one fails, send some requests one way and the rest another, and split traffic between models by weight. The app still sends only "model": "support".

Three kinds of step

A route is a tree of steps. A route of one model is the alias as it always was.

StepWhat it does
ModelCalls a model of the project's catalogue. Its fallback, a step of any kind, takes over when the model fails for a reason the step lists
ConditionTakes then when its rules hold for the request, otherwise else
SplitSends each request down one of 2 to 8 branches, by weights that add up to 100

A route has at most 32 steps and is at most 12 deep, and every model it names must be in the project's catalogue.

Fallbacks

When a model step fails, Kaeso moves on to its fallback if the step's fall_back_on lists the reason:

ReasonFailures
unavailableThe provider can't be reached, times out, is overloaded or answers 5xx; it refuses the project's key or doesn't serve the model
limitedThe provider's rate limit or quota, or a Kaeso limit or budget for the step's model
unsupportedThe model can't take something the request uses, such as images, PDFs, tools or a JSON schema. Kaeso knows this from the catalogue before it sends anything
rejectedThe provider refuses the request itself, such as a context that is too long

A step falls back on the first three unless it lists others. Add rejected only for a fallback that can take what the first model refused, such as a model with a longer context. A step's timeout_seconds sets how long Kaeso waits for the first byte before it moves on.

Once the provider has started to answer, nothing falls back: a stream that breaks off later ends with an error event, as it would without a route.

Failures that last are skipped

A refused key, an empty balance or a rate limit doesn't fix itself in the next second. When a step fails for a reason that lasts, later requests skip it for a while and go straight to the next step: 5 minutes for a refused key or an empty balance, as long as Retry-After says for a rate limit, 30 seconds for a timeout or a 5xx. An answer from the model ends the wait. Every gateway of an install shares this through Valkey, so a step one gateway saw fail is skipped by all of them.

Conditions

A condition reads facts about the request, with 1 to 10 rules joined by all or any:

  • A header equals, contains, or is present or absent. Send x-plan: enterprise from your app, and route paying customers to a larger model.
  • The format of the app, such as anthropic_messages or gemini.
  • A feature the request uses: images, PDFs, tools, reasoning, structured output or streaming.
  • Input tokens, at least or below a number, to send long prompts to a model with a longer context.
  • The caller: the API key, member or service account that sent it.

Routes never read headers that carry credentials, such as authorization or x-api-key.

Splits

A split draws a number for each request and takes the branch its weight covers. How it draws decides what the split is good for:

  • by: request draws anew every time, for spreading load.
  • by: caller keeps each API key, member or service account on one branch. Use it for A/B tests, so one app sees one model. Changing the weights later moves only the callers whose draw crosses a new boundary.
  • by: header keeps each value of a header on one branch, such as x-user-id, so each of your users sees one model behind a single API key.

A route, read from top to bottom

The live demo (opens in a new tab) has a support alias that shows all three in one route. Requests with x-plan: enterprise, or with at least 100,000 input tokens, go to Claude Sonnet 5.5, with GPT-6 Sol as its fallback. Every other request is split by caller: about four in five get Claude Haiku 5.5, with GPT-5.4 mini as its fallback, and the rest get GPT-5.4 mini.

The route of the support alias in the editor: a condition on x-plan sends enterprise requests to Claude Sonnet 5.5 with GPT-6 Sol as its fallback, and a split by caller sends the rest to Claude Haiku 5.5 or GPT-5.4 miniThe route of the support alias in the editor: a condition on x-plan sends enterprise requests to Claude Sonnet 5.5 with GPT-6 Sol as its fallback, and a split by caller sends the rest to Claude Haiku 5.5 or GPT-5.4 mini
Fig. 01The support alias in the demo: a condition, a split by caller and fallbacks.

Build and test it in the console

Open Models › Aliases in a project and create an alias, or open one. The route editor is a full page: start from one of its templates, or build the tree step by step. Its Test tab sends a request through the draft, before you save it, and colors each step by its answer. A trace shows each condition with the rules that held, the number each split drew and the branch it took, and every model the request would try, in order.

The same trace is on the Platform API, so a script or a CI job can check a route before it goes live, and agents get it as a read-only tool over MCP.

What apps and logs see

  • kaeso-model and kaeso-provider name the step that answered.
  • kaeso-route names every step the request reached, in order, with its model, status and time: "route";model="anthropic/claude-sonnet-5-5";status=529;dur=1204, "route.fallback";model="openai/gpt-6-sol";status=200.
  • The request log keeps the alias as the requested model and lists every step tried or skipped, with its status and error.
  • Usage and cost are the answering step's. Failed steps cost nothing in Kaeso, and a step that fails gives back what it held of a budget before the next one asks.
  • kaeso_alias_fallbacks_total{provider, reason} counts fallbacks and skips for your dashboards.

A key that may not use a step's model skips it without learning its name, so a restricted key learns nothing about the route.

Next steps

By Kaeso Team

On this page

Own the gateway.Own the record.

curl -fsSLhttps://kaeso.app/install.sh | sh