Models API

This page is for developers who want to read and adjust the models their organization can use, or run a model on their own cloud account, from their own code. For example, a script can list which models an agent names that can’t be used right now. In the app, this is the Models page. For the concepts, see Models, Routing and Your cloud account.

REST reference for models and for running a model on your own cloud account. Every response is wrapped in { success, data }; every route is scoped to the caller’s current organization.

Models

Models arrive from connected providers by themselves (POST /llm-providers/connect); a provider’s models are selectable once its key check passes. These routes read and adjust them.

Every model in a response carries computed fields: selectable (usable right now), allowed (its connection offers it; see Which models a connection offers), effectivePricing (the override when set, else the feed price) and, on the list, isNew (its connection started listing it within the last week).

Models an agent cannot use

GET /models/agents/:agentId/issues answers, for an agent you can read, every model it names (its model, roles, steps, checkers and team) that cannot be used now:

{
  "success": true,
  "data": [
    { "model": "Qwen/Qwen3-32B", "modelName": "Qwen/Qwen3-32B", "providerId": "7c1e...", "connectionName": "HF - everything", "reason": "The provider no longer lists it.", "where": ["model", "role Checker"], "since": "2026-09-29T08:00:00.000Z" }
  ]
}

connectionName is null when the connection is not yours to see. The list is empty once every model is usable again.

Add a model by hand

For an id the provider serves but does not list. It is checked when its provider is.

curl -X POST https://api.almyty.com/models \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Qwen3 32B on our vLLM",
    "providerId": "7c1e...",
    "vendorModelId": "qwen3-32b",
    "privacyTier": "private_cloud",
    "region": "eu-central",
    "capabilities": { "tools": true },
    "contextLength": 32768
  }'

Your own server

Any OpenAI-compatible server (vLLM, Ollama, TGI, llama.cpp, LiteLLM) is a custom provider. Connect it with POST /llm-providers/connect, type: "custom", the server’s base URL in configuration.apiUrl and, if it checks one, its key in configuration.apiKey. What the server lists at GET <base>/models becomes its models, usable once the check passes.

Validate

For an endpoint with no provider row (a provider’s models are covered by its key check, and a model on your cloud account is checked by itself when it reaches ready):

curl -X POST https://api.almyty.com/models/4f2a.../validate \
  -H "Authorization: Bearer $TOKEN"

{
  "success": true,
  "data": {
    "passed": true,
    "latencyMs": 812,
    "model": { "id": "4f2a...", "validationStatus": "passed", "selectable": true, "...": "..." }
  }
}

A failed run answers with success: false, passed: false and error, and records the error on the model.

Sync

POST /models/sync with no body returns a per-provider summary; with { providerId } the summary for that provider: created and reinstated models, retired models, skipped count, and error per provider where the vendor could not be asked. The same read happens on connect, on every key check and every six hours without anyone calling it.

Model fields

id, name, providerId, providerType, vendorModelId, endpointRef, base, modelVersionId, capabilities, contextLength, pricing, pricingSource (feed:litellm, feed:openrouter, native, adapter, manual, unpriced), pricingFetchedAt, pricingOverride, measuredLatencyMs ({ p50, p95, updatedAt }), privacyTier, region, status (active, inactive, error, deploying), validationStatus (never, passed, failed), lastValidatedAt, lastValidationError, metadata, createdAt, updatedAt, plus the computed allowed, selectable, effectivePricing and isNew.

For a model on your cloud account, endpointRef carries url, providerType and deploymentId (the id under /model-deployments below).

Routing

A routing policy travels inside an LLM Call node’s config, an autonomous agent’s modelConfig.routing, or the organization’s settings.defaultRouting; there is no routing endpoint of its own. The shape and every field are on Routing. The organization default is written with:

curl -X PATCH https://api.almyty.com/organizations/$ORG \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "settings": { "defaultRouting": { "objective": "cheapest", "privacyTier": "private_cloud" } } }'

null clears it; other settings keys are left as they are. The attribution of a routed answer (routing on the chat response, on nodeResults[nodeId], and on autonomous step output) is { modelId, modelVersionId, vendorModelId, providerId, rationale, attempt, tried, rejected }.

Your cloud account

A model on your own cloud account is created, sized and shut down through the /model-deployments routes; the clouds are listed by /model-adapters. Those are the API’s names for it: each cloud is an adapter with a providerType key, and each model running on one is a row under /model-deployments.

GET /model-adapters carries modelSchemes per cloud: every source it accepts, as literal prefixes (hf://, s3://, bedrock://, …). A form filters both ways with it: the clouds that can run the source you have, or the sources the cloud you picked will take. configSchema is a JSON schema for the cloud’s providerConfig; properties marked "x-secret": true are encrypted at rest and never returned.

Start a model from a Hugging Face repository:

curl -X POST https://api.almyty.com/model-deployments \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hf://Qwen/Qwen3-0.6B",
    "providerType": "huggingface-endpoints",
    "name": "Qwen3 0.6B",
    "desired": { "replicas": 1, "region": "eu-west-1", "privacyTier": "private_cloud" },
    "providerConfig": { "token": "hf_...", "namespace": "acme", "region": "eu-west-1", "instanceType": "nvidia-a10g" },
    "budgetId": "5d3f..."
  }'

Or a model the cloud already hosts:

{ "model": "fireworks://accounts/acme/models/qwen3-tuned", "providerType": "fireworks" }

Exactly one of model or modelVersionId is required; naming neither is MODEL_REQUIRED. A source the cloud can’t read is refused with ADAPTER_UNSUPPORTED_SOURCE and an accepts array of the schemes it does take. An s3:// source in an organization with no bucket connection is REGISTRY_NOT_CONNECTED.

The model exists at once. Unless modelId names an existing one, POST /model-deployments creates the model in the list straight away, with status: deploying (not selectable), and links it. When the endpoint is ready the model gets endpointRef.url and endpointRef.deploymentId, turns active, takes the region the cloud reports (or the one asked for), and is priced from the cloud’s cost report (pricingSource: adapter) where the cloud reports a per-token price and no override is set. Validate it and it is selectable.

A row in a response carries state (pending, deploying, ready, degraded, scaling, tearing_down, torn_down, orphaned, failed), modelRef or modelVersionId, desired, actual (state, url, replicas, hardware, region, message, spentCents, ratePerHourCents), externalRef, lastReconcileAt, lastError, budgetId, and providerConfig with every secret shown as ********. scale and teardown change the wanted state only; the reconcile sweep (MODEL_RECONCILE_CRON, every two minutes, plus a run right after each change) is the only thing that talks to the cloud.

Cloud errors carry ADAPTER_UNKNOWN, ADAPTER_UNSUPPORTED_SOURCE (the source’s scheme is not one this cloud reads), ADAPTER_UNSUPPORTED_OPERATION (the cloud cannot do this at all), ADAPTER_UNSUPPORTED_ARCHITECTURE, ADAPTER_PREVIEW_NOT_ENABLED, ADAPTER_AUTH, ADAPTER_ERROR.

No cloud has yet been run against a real account; see Your cloud account for what each one actually accepts.

Registered versions

Optional, for power users. Nothing needs a registered version: POST /model-deployments takes the source directly and pins a Hugging Face repository for you. Register one when you want a manifest digest, lineage and evaluation history recorded against your own weights, then pass its id as modelVersionId.

curl -X POST https://api.almyty.com/model-versions \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "name": "qwen3-14b-support-ft", "registryUri": "s3://acme-models/qwen3-14b-support@e3b0c442" }'

Registering an s3:// version reads and validates almyty-manifest.json at the URI and fills base, size, digest and quantizations from it; an unreadable manifest is an error. For every other scheme the manifest is optional, and base is required when it is absent. The layout under the prefix is the manifest plus plain safetensors (model.safetensors, or sharded model-00001-of-0000N.safetensors) and the tokenizer files it references:

{
  "schemaVersion": 1,
  "base": "qwen3-14b",
  "tokenizer": "s3://acme-models/qwen3-14b-support@e3b0c442/tokenizer",
  "chatTemplate": "chatml",
  "license": "Apache-2.0",
  "created": "2026-09-08T09:00:00Z",
  "files": [{ "path": "model.safetensors", "sizeBytes": 29000000000, "sha256": "..." }],
  "quantizations": ["bf16", "awq-int4"],
  "lineage": { "parentVersionId": "..." }
}

Required: schemaVersion (1), base, tokenizer, license, created and at least one entry in files with relative paths. The manifest’s canonical SHA-256 is recorded as manifestSha; the sum of files[].sizeBytes becomes sizeBytes.

Errors: REGISTRY_NOT_CONNECTED (no bucket connection for the organization, on an s3:// URI), VERSION_IN_USE on delete while a model that is not shut down references the version.

Audit actions

model_registered, model_validated, model_price_updated, model_routed, model_route_escalated, state transitions of models on your clouds, model_deployment_budget_stop, model_deployment_orphan_teardown.