Connect a provider
To use an AI model in almyty, you connect the provider that runs it, such as OpenAI, Anthropic or Mistral, with your own key. Once connected, every agent, workflow step and AI-powered tool can use that provider’s models, and you pay the provider directly at their prices. For example: connect OpenAI once, and your support agent can answer with GPT-4o today and switch to another model tomorrow.
A connection is a named key (or the address of a server you run, Ollama, or a cloud account). Manage connections on Models; their keys also appear in Credentials. Connecting one makes its models available in every model picker. You can connect the same provider more than once, each with its own name, key and models.
In the dashboard
- Open Models in the sidebar and click Connect a provider, or click Add a connection at the bottom of any model chooser to do the same without leaving the page.
- Choose your provider from the searchable Provider list, including Your own server and Ollama.
- Name the connection. It starts as the provider’s name; give it one that says what it is for, such as “OpenAI - research team”.
- Paste the API key, or click Use a saved key instead and pick a key already in Credentials (or add one there without leaving the page). Your own server asks for its address and an optional key; Ollama asks for an Ollama Cloud key, or the address of an Ollama you run (see Ollama). A provider that needs something more asks only for that: AWS Bedrock the region, Azure the resource and the model name in Azure, Vertex AI the project and the model, RunPod the endpoint. A provider that does not publish its list of models (Qwen, Z.ai, Fireworks, SambaNova, Doubao Ark, Spark, Vertex AI) asks which model to use.
- Who can use it reads “everyone in your organization”. Click Change to make it yours alone or a team’s.
- Click Save.
almyty checks the key with one real call. If the check fails you see one plain line, such as “OpenAI rejected this key.”, next to the key field, with the provider’s own words under Details and a link to get a key. Nothing is saved until the key works.
When it works, the form opens the new connection under Models, or returns to the page that sent you. All models are offered initially; choose which ones it may offer on its connection page.
A connection’s page
Click a connection on the Models page, or open its key from Credentials. The header shows its name (the pencil renames it), whether its key works and which models it offers, with Check again to re-check the key and fetch its model list again.
When the provider refuses a connection’s key, almyty turns the connection off. Its page then says This connection is inactive, gives the reason from its last check, and offers Replace key and Check again right there. While it is off, its models are not offered anywhere; once a check passes again, it is back on.
- Models: every model its key reaches with price, how much text it can read at once, and status. Tick the ones it offers; Allow new models automatically says whether a model the provider adds later is offered too. With the switch off and one model ticked, the key serves that model and nothing else. With none ticked the connection is paused. A model an agent uses cannot be unticked until those agents use another one; saving names them. See Which models a connection offers.
- Settings: Replace key, who can use it, the default model it answers with when a step names none, and Advanced: your own price, reading limit, privacy level, region and abilities per model, an admin key for matching usage with the provider’s bill (OpenAI, Anthropic), and default answer settings. Remove connection asks once, naming the agents that use its models; their owners are told afterwards that the model is no longer available. Its models stay in past runs.
- Open models on this account, on a cloud account connection: see Your cloud account.
The connection’s key is listed on Credentials under the connection’s name; renaming one renames the other. Every key picker offers it too, since the same key serves the provider’s other services.
Ollama
Ollama runs open models, either on Ollama’s own hosted service or on a machine you run. Choosing Ollama asks Where Ollama runs, with Ollama Cloud picked first.
- Ollama Cloud is Ollama’s hosted service. Paste an API key from ollama.com/settings/keys ; nothing else is asked. You get Ollama’s hosted models (large open models like
gpt-oss:120borqwen3.5:397b, without hardware of your own). Ollama bills them by plan, so their price reads “Price unknown” rather than zero; track that spend in your Ollama account. - Your own server is an Ollama you run. Give its address (for example
http://build-box:11434) and, only if it sits behind a proxy that checks one, a key. Its models are the ones you pulled onto it, and they are free and marked as local. On hosted almyty the server must be reachable at a public address.
Using the models
Every place that needs a model (an agent’s work mode, a workflow Model call step, a role) picks it from one search box listing every model your connections offer, grouped by connection, with each model’s price, or chooses Automatic to let almyty pick the cheapest model that fits. Add a connection at the bottom of that list connects another provider in place. See Choosing a model, Routing and Agents.
almyty records the tokens and cost of every AI call; see the totals per provider in Analytics.
From your AI assistant
almyty can itself be controlled from an AI assistant such as Claude Code or Cursor, which can add and list providers for you without opening the dashboard. See Control almyty via MCP.
For developers
Supported providers
almyty speaks to 33 providers out of the box, plus any OpenAI-compatible endpoint you point it at.
| Provider | Type | API format | Serves |
|---|---|---|---|
| Anthropic | anthropic | Anthropic messages (preferred) | Claude Opus, Sonnet and Haiku |
| AWS Bedrock | aws_bedrock | OpenAI chat completions (preferred) | Bedrock models in your own AWS account |
| Azure AI Foundry | azure_ai_foundry | OpenAI chat completions (preferred) | Foundry model deployments in your own Azure subscription |
| Baidu ERNIE (Qianfan) | qianfan | OpenAI chat completions (preferred) | ERNIE 5.1 and the open models Qianfan hosts |
| Baseten | baseten | OpenAI chat completions (preferred) | Open models, and your own weights or container |
| ByteDance Doubao (Ark) | volcengine | OpenAI chat completions (preferred) | Doubao Seed 2.0, and DeepSeek on Ark |
| Cerebras | cerebras | OpenAI chat completions (preferred) | Open models on wafer-scale hardware |
| Cohere | cohere | OpenAI chat completions (preferred) | Command and the Cohere retrieval models |
| DeepInfra | deepinfra | OpenAI chat completions (preferred) | Open models per token, and your own weights on dedicated GPUs |
| DeepSeek | deepseek | OpenAI chat completions (preferred) | DeepSeek V4 and the reasoning line |
| DigitalOcean Gradient | digitalocean | OpenAI chat completions (preferred) | Open models on DigitalOcean serverless inference |
| Fireworks AI | fireworks | OpenAI chat completions (preferred) | Open models and your own fine-tunes on dedicated GPUs |
| Google Gemini | google | Gemini generateContent (preferred) | Gemini Pro and Flash on the Gemini Developer API |
| Groq | groq | OpenAI chat completions (preferred) | Open models at very low latency |
| iFlytek Spark | spark | OpenAI chat completions (preferred) | Spark X2, X1.5 and the 4.0 Ultra line |
| MiniMax | minimax | OpenAI chat completions (preferred) | MiniMax M3 and the M2 family |
| Mistral AI | mistral | OpenAI chat completions (preferred) | Mistral Large, Medium and Codestral |
| Modal | modal | OpenAI chat completions (preferred) | Your own endpoints on Modal |
| Moonshot (Kimi) | moonshot | OpenAI chat completions (preferred) | Kimi K3 and the K2.7 coding models |
| Nebius Token Factory | nebius | OpenAI chat completions (preferred) | Open models, and LoRA adapters on curated bases |
| Novita | novita | OpenAI chat completions (preferred) | Open models, and any Hugging Face repo on a dedicated endpoint |
| OpenAI | openai | OpenAI chat completions (preferred) | GPT-5 and the o-series |
| OpenRouter | openrouter | OpenAI chat completions (preferred) | One key across hundreds of models and providers |
| Qwen (QwenCloud) | qwen | OpenAI chat completions (preferred) | Qwen3 Max, Plus and Flash on QwenCloud |
| RunPod | runpod | OpenAI chat completions (preferred) | Your own serverless vLLM worker |
| SambaNova | sambanova | OpenAI chat completions (preferred) | Open models on RDU hardware |
| Straitly | straitly | OpenAI chat completions (preferred) | One key across 177 models, with cashback instead of a markup |
| Tencent Hunyuan (TokenHub) | hunyuan | OpenAI chat completions (preferred) | Hunyuan, plus DeepSeek, GLM, Kimi and MiniMax on TokenHub |
| Together AI | together | OpenAI chat completions (preferred) | A large open-model catalog, and your own fine-tunes |
| Upstage Solar | upstage | OpenAI chat completions (preferred) | Solar Pro 4 and Solar Mini |
| Writer (Palmyra) | writer | OpenAI chat completions (preferred) | Palmyra X6, X5 and the domain models |
| xAI (Grok) | xai | OpenAI chat completions (preferred) | Grok 4 and the Grok code models |
| Z.ai (GLM) | zai | OpenAI chat completions (preferred) | GLM 5.3 and the GLM coding models |
Each native type ships with the right default base URL; set configuration.apiUrl to point at a compatible proxy instead. For custom (Your own server), set configuration.apiUrl to the server’s OpenAI-compatible endpoint; see Your own server. The configuration.apiKey field accepts whatever token the provider expects.
Types not in the table above
Four more provider types ship and appear in the Provider list, but have no row above: the table is generated from the provider registry, and these four still use an older per-vendor code path. The vendor reference gives the base URL, sign-in and model-listing details for each.
| Type | Vendor | Notes |
|---|---|---|
azure_openai | Azure OpenAI | Base resolves from your resource name; authenticates with an api-key header. Not the same product as azure_ai_foundry, which does have a row above. |
vertex_ai | Google Vertex AI | Authenticates with a short-lived OAuth token minted per call from a service account. Not the same product as google, the Gemini Developer API. |
huggingface | Hugging Face | Bearer hf_...; the model list reports per-provider tool support and pricing. |
perplexity | Perplexity | Bearer key on a Responses-shaped surface, with an unauthenticated model list. |
Configuration reference
| Field | Type | Required | Description |
|---|---|---|---|
name | string | yes | Display name, up to 100 characters |
type | enum | yes | One of the provider types listed above |
configuration | object | yes | Everything below nests inside it |
description | string | no | Up to 1000 characters |
configuration.apiKey | string | no | Provider key, encrypted at rest. Required for Ollama Cloud; optional for an Ollama you run |
configuration.apiUrl | string | no | Overrides the type’s default base URL; the field a compatible proxy goes in. Required for custom |
configuration.model | string | no | Default model. Left blank, it resolves to the vendor’s current default by family rather than a hardcoded id |
configuration.maxTokens | number | no | Sent when set; otherwise the vendor’s own default applies |
configuration.temperature | number | no | Sent when set; otherwise the vendor’s own default applies |
configuration.topP | number | no | Sent when set; otherwise the vendor’s own default applies |
allowNewModels | boolean | no | Default true: models the provider adds later are offered. false: only allowedModels |
hiddenModels | string[] | no | Model ids unticked while allowNewModels is on |
allowedModels | string[] | no | Model ids ticked while allowNewModels is off; at least one |
There is no top-level baseUrl, apiKey or model field: all three nest under configuration, and the base-URL override is named apiUrl. See the Providers API for the full body.
Ollama details
Ollama Cloud models are listed from GET /api/tags and called through the OpenAI-compatible /v1 surface with the key as a Bearer token; over the API, configuration.apiUrl is https://ollama.com and a key is required. An Ollama you run is called on its OpenAI-compatible /v1 endpoint for chat, streaming and tool calling; its models are read from the native GET /api/tags, and memory embeddings use the native POST /api/embed. Its models get the privacy tier local. Because such a server usually listens on localhost or a private network, almyty refuses those addresses unless the install opts in with OLLAMA_ALLOW_PRIVATE_URLS=true on the backend (see Self-hosting). An ollama connection made over the API with no apiUrl is Ollama Cloud (https://ollama.com), and needs a key.
Cost tracking
The cost of each call is its token counts times the model’s price: a price set on the connection wins, then the live price feed (see Pricing), then a built-in offline price table used only when the feed has no entry. Ollama usage is always recorded at $0 (an Ollama you run has no per-token price; Ollama Cloud bills by plan in your Ollama account). See Cost governance for reconciliation against provider actuals, which only OpenAI and Anthropic support.
Memory embeddings
The Memory module embeds content with the organization’s own providers. When several are configured, OpenAI (text-embedding-3-small) is preferred over Mistral (mistral-embed), then Ollama (nomic-embed-text by default via the native /api/embed endpoint, overridable with configuration.embeddingModel); with none, a deterministic local fallback keeps memory searchable. Each stored memory records the model that embedded it (dimensions vary per model), and vector search only compares like with like.
Links and API
Other setup pages can link to /models/providers/new?type=openai (open with that provider chosen) and ?returnTo= to come back when done. A connection’s page is /models/providers/:id. The MCP control plane exposes add_provider and list_providers.
Connecting (POST /llm-providers/connect), provider CRUD, key checks and chat sessions are available over REST. See the Providers API reference.