Routing

Routing lets almyty pick the model for you, call by call, instead of you naming one. You say what matters (cheapest, fastest, or a model you prefer first) and any limits (keep it private, keep it in the EU, it must be able to use tools, stay under a spend limit), and almyty chooses the best model that fits from the ones your connections offer. If that model’s provider is down, it moves on to the next one, and every answer records which model was picked and why.

For example: a support agent’s first draft set to Cheapest, limited to models in the EU. almyty picks the cheapest EU model today, and if a cheaper one appears next month, it uses that without anyone touching the agent.

Turn it on

Wherever you choose a model (an agent’s work-mode slots, a Model call step in a workflow), pick Automatic at the top of the list. It reads Automatic: the cheapest model that fits until you change it. Open Automatic settings under it to set:

What happens on each call

1. almyty lists every model you can use, and leaves out the ones that break a limit, noting why (“lacks tools”, “region unknown”, “over the spend limit”).
2. It orders the rest by your objective, or by your fallbacks. Models with no known price or speed go last.
3. It calls the first. If that provider fails for a reason that is not the request’s fault (an outage, a retired model, a used-up quota, a refused key), it tries the next. If the request itself is the problem (too long, malformed), it stops, because the next model would fail the same way.

A streamed answer (one that appears word by word) cannot switch models halfway, so it uses the first model only.

See what was chosen

Every automatic choice is recorded: on each step of a run (which model answered, the reason, and the models it passed over), and in the audit log. So you can always answer “why did it use that model?”.

When no model fits

If no model satisfies the limits, the call fails and says how many models were left out and why. This often means the model you relied on is no longer usable; the Models page shows each model’s status.

Related

• Models: connections, prices and speed
• Cost governance: budgets
• Autonomous agents: checkers and work modes

For developers

A step carries a routing policy instead of a provider:

{
  "routing": {
    "objective": "cheapest",
    "privacyTier": "private_cloud",
    "regions": ["eu-central"],
    "capabilities": { "tools": true },
    "fallbackChain": ["<card id or vendor model id>", "..."],
    "pinnedModel": "<card id or vendor model id>",
    "budgetHeadroomCents": 500,
    "escalation": { "onVerifyFail": "next-candidate", "maxEscalations": 2 }
  }
}

Where a policy goes

• An LLM Call node (Model call in the builder) in a workflow agent takes routing in place of providerId.
• An autonomous agent takes modelConfig.routing, which replaces providerId for every step of the run.
• The organization default (settings.defaultRouting) applies to any LLM Call node that names neither a provider nor a policy. It has no dashboard control.

A policy on the node always wins over the organization default. A node with neither, in an organization without a default, fails with a message that says so.

The policy

The organization default has the same shape: PATCH /organizations/:id { settings: { defaultRouting } } sets it, null clears it, and other settings keys are left as they are. The update validates every field. The executor caches the default for 30 seconds per organization, so a change applies to the next run, not the one in flight.

How a plan is built

Selection is pure: the same cards and the same policy always give the same plan.

1. Filter. A card is eligible when it is selectable, its privacy tier is at or below the ceiling, its region is known and allowed, it has every required capability, and its price fits the budget headroom. Each card that fails records a reason (privacy tier public above ceiling private_cloud, region unknown, policy requires one of eu-central, lacks tools, over the spend limit, and so on).
2. Order. With a fallbackChain, the plan is the chain in the order given. Otherwise cheapest sorts by blended feed price (three quarters input, one quarter output, per million tokens), fastest by measured p50 latency, and pinned moves the pinned card to the front of the price order. Unpriced or unmeasured cards sort last.
3. Resolve. Each candidate is paired with something callable: its provider, or, for a model on your cloud account, a transient provider built from its endpoint URL and the cloud’s token. A card whose provider is unhealthy or inactive is rejected at this point with the provider’s status as the reason.

Walking the plan

The chat runner calls the first candidate. If the call fails for a reason that is not the request’s fault (a retired model, quota, an outage, an authentication problem on that provider), the candidate is recorded in tried with the reason and the next one is called. A request-shaped failure (400, 413, 422) or a caller abort stops the walk and surfaces the error. When every candidate fails, the last error is returned with the code ROUTE_EXHAUSTED and the tried list.

A streamed request takes the head of the plan and stamps the same attribution with attempt: 1. If the head fails before the first token, the request fails; there is no fallback inside a stream.

Attribution

{
  "modelId": "4f2a...",
  "modelVersionId": null,
  "vendorModelId": "qwen3-32b",
  "providerId": "7c1e...",
  "rationale": "cheapest ($0.41/M blended), rank 1",
  "attempt": 1,
  "tried": [],
  "rejected": [
    { "modelId": "9b0d...", "reason": "privacy tier public above ceiling private_cloud" }
  ]
}

• Node results. On a workflow run, nodeResults[<nodeId>].routing shows which card served each LLM Call node and why.
• Run steps. On an autonomous run, each step that called a model carries the same object, and the chat response itself carries it as routing.
• Audit log. Every routed call writes a model_routed entry with the model, version, rationale and attempt, next to model_registered, model_validated and model_price_updated for the catalog itself.

Verify escalation

Normally the plan moves on only when a candidate fails to answer. With verify escalation, it also moves on when a candidate answered but a verifier rejected the answer. It is off by default; set MODEL_ROUTER_VERIFY_ESCALATION=true on the API to enable it, then add an escalation block to the policy:

On a rejection the run re-issues the request with the candidates already tried skipped; those stay visible in rejected with the reason skipped after verify escalation. The run’s working memory carries the adjusted policy and the count, and the run emits a route.escalated event. The escalation stops when the plan has no further candidate, when maxEscalations is spent, or when the run was cancelled. With the flag off, every failed verdict is a stop.

Errors

npx @almyty/models list shows every card with why it is not selectable.