Models CLI

This page is for developers who prefer the terminal. The everyday tasks (connecting providers, choosing which models a connection offers, starting a model on your cloud account) are also on the Models and Credentials pages in the dashboard; see Models.

@almyty/models is a command-line tool that manages your models: list them and say why one cannot be picked, adjust them, run a model on your own cloud account, and watch what it costs. It shares the login of every other almyty command-line tool.


$ npx @almyty/auth login
$ npx @almyty/models --help

The umbrella CLI exposes the same commands as almyty models <command>. Every command takes --json for raw output, ALMYTY_TOKEN to skip the stored login, and ALMYTY_URL to point at another almyty server.

Models

Models come from connected providers by themselves, usable once the provider’s key check passes. List every model, or only the ones the router may pick:


$ npx @almyty/models list
Claude Sonnet  [claude-sonnet-5]  public  $3/$15 per M (feed:litellm)
    9b0d...  selectable
Qwen3 32B on our vLLM  [qwen3-32b]  private_cloud/eu-central  unpriced
    4f2a...  not selectable: waiting for its provider's key check (check its connection under Credentials)
 
$ npx @almyty/models list --selectable

Show one model:


$ npx @almyty/models get 9b0d...

Add a model a provider serves but does not list, naming the vendor id yourself. It is usable when its provider is:


$ npx @almyty/models register \
    --name "Qwen3 32B on our vLLM" \
    --provider 7c1e... \
    --model qwen3-32b \
    --tier private_cloud \
    --region eu-central \
    --context 32768
Registered.
Qwen3 32B on our vLLM  [qwen3-32b]  private_cloud/eu-central  unpriced
    4f2a...  selectable

Your own server (vLLM, Ollama, TGI, llama.cpp, LiteLLM) works the same way: it is a custom provider, connected once in the dashboard or with POST /llm-providers/connect (see Your own server), and --provider names it. Its key lives with the provider, so no key is ever passed here.

Read what a provider currently lists again (this also happens by itself every six hours):


$ npx @almyty/models sync 7c1e...
12 card(s) created, 3 already present.

Check one model with a real short call. A provider’s key check already covers its models; this is for an endpoint with no provider row. The exit code is 1 when it fails:


$ npx @almyty/models validate 4f2a...
Passed in 812 ms.
Qwen3 32B on our vLLM  [qwen3-32b]  private_cloud/eu-central  unpriced
    4f2a...  selectable
 
$ npx @almyty/models validate 4f2a... ; echo $?
Failed: model qwen3-32b is not served by this endpoint
1

Remove a model:


$ npx @almyty/models delete 4f2a...
Deleted.

Flag	Commands	Meaning
`--provider`	register	The provider’s id
`--model`	register	The vendor model id sent on the wire. Not the model source that `host` takes below
`--tier`	register	`public`, `private_cloud` or `local`
`--region`	register	Region label used by routing allowlists. A model with no region is excluded by any policy that sets one
`--context`	register	Context length in tokens
`--selectable`	list	Only models the router may pick

Your cloud account

These commands run a model on your own cloud account: host starts a model, hosted lists the ones on your clouds, and --adapter takes the cloud’s providerType. deploy, deployments and deployment <id> also work, as other names for host, hosted and hosted <id>, matching the API’s /model-deployments routes.

List the clouds and what each can do:


$ npx @almyty/models adapters
modal  Modal
    architectures="any" lora="merged" serverless=true dedicated=false scaleToZero=true regions=[] registrySources=["hub"]
huggingface-endpoints  Hugging Face Inference Endpoints
    architectures="any" lora="merged" serverless=false dedicated=true scaleToZero=true regions=["us-east-1","us-west-2","eu-west-1","eu-central-1","ap-southeast-1"] registrySources=["hub"]
...

--json includes each cloud’s modelSchemes (every source it accepts, as literal prefixes) and its config schema, the same data the Start a model form renders from.

The model source is the positional argument. The cloud’s account details carry secrets, so they come from a file, from stdin or from a saved connection, never from argv. Start a model from a Hugging Face repository:


$ cat > hf.json <<'JSON'
{"token":"hf_...","namespace":"acme","region":"eu-west-1","instanceType":"nvidia-a10g"}
JSON
$ npx @almyty/models host hf://Qwen/Qwen3-0.6B \
    --adapter huggingface-endpoints \
    --config-file hf.json \
    --desired '{"replicas":1,"region":"eu","privacyTier":"private_cloud"}' \
    --budget 5d3f...
Queued.
c07b...  huggingface-endpoints  pending  replicas ?/1

The model is in list at once, not yet selectable; almyty checks it with one short call when the endpoint is ready, and it becomes selectable if that passes (validate <id> runs it again). Better still, connect the account once and name the connection, so no key reaches this machine’s filesystem at all:


$ npx @almyty/models host hf://Qwen/Qwen3-0.6B \
    --adapter huggingface-endpoints \
    --credential 9b4c... \
    --desired '{"replicas":1,"region":"eu","privacyTier":"private_cloud"}'

Or a model the cloud already hosts:


$ npx @almyty/models host bedrock://anthropic.claude-3-sonnet-20240229-v1:0 --adapter aws-bedrock-import

--config '<json>' still exists, but host screens it against the cloud’s config schema and refuses it the moment it carries a field the cloud marks secret: token for huggingface-endpoints, tokenId and tokenSecret for modal, apiKey for fireworks and the rest. argv is written to your shell history, is echoed by most CI runners and is readable by any process through ps, so a secret that travelled that way is disclosed. --config is also refused outright when the cloud catalog cannot be read, because then almyty does not know which of the fields are secret and will not guess. Run adapters to see which fields each one marks secret. --config is fine for the fields that are not secret, such as namespace, region, instanceType or gpu.

Flag	Meaning
`<model>` (positional)	The model source: `hf://org/repo` (pinned to its exact commit for you; `@sha` pins one yourself), `s3://bucket/prefix@etag`, `gs://bucket/prefix@gen`, `file:///path@sha`, or a model the cloud already hosts: `bedrock://`, `sagemaker://`, `vertex://`, `foundry://`, `azureml://`, `fireworks://`, `together://`, `baseten://`. Required unless `--model-version` is given
`--adapter`	The cloud’s `providerType`, from `adapters` (required)
`--config-file <path>`	The cloud’s config read from a JSON file. The way to pass secret fields
`--config-stdin`	The cloud’s config read from stdin. Same, for a secret that should not touch disk
`--config '<json>'`	The cloud’s config inline. Refused if it carries a field the cloud marks secret, or if the catalog could not be read to check
`--credential <id>`	A credential added with `almyty credentials add`; the cloud’s secrets are read from there instead of from any config you pass
`--desired`	JSON: `hardware`, `replicas`, `minScale`, `maxScale`, `quantization`, `region`, `privacyTier`
`--budget`	A spend budget; reaching it stops the model (scales it to zero)
`--card`	An existing model to attach instead of adding a new one; it gets the endpoint once ready and keeps being priced from the cloud
`--base`	The model family, when the cloud checks it and the source does not carry one
`--model-version`	A registered version instead of the positional source; see Registered versions

adapters lists what each cloud accepts; naming a source the chosen cloud can’t read is refused before anything runs.

Watch state and spend:


$ npx @almyty/models hosted
c07b...  huggingface-endpoints  ready  replicas 1/1  spent 2.40 USD
e18a...  huggingface-endpoints  failed  replicas 0/1  spent 0.00 USD
    quota exceeded for nvidia-a10g in eu-west-1

hosted <id> shows one. Change the replica count (0 stops it) or shut it down. Both record what you want; almyty applies it at the cloud within two minutes:


$ npx @almyty/models scale c07b... 0
c07b...  huggingface-endpoints  scaling  replicas 1/0  spent 2.40 USD
 
$ npx @almyty/models teardown c07b...
c07b...  huggingface-endpoints  tearing_down  replicas 1/0  spent 2.40 USD

teardown removes the endpoint at the cloud. Weights are untouched: a Hugging Face repository, your bucket and a model the cloud hosts all stay where they are.

Registered versions

Optional, and not needed to run anything: host takes the source directly and pins a Hugging Face repository for you. Register a version when you want a manifest digest, lineage and evaluation history recorded against your own weights, then pass --model-version to host.


$ npx @almyty/models versions
qwen3-14b-support-ft  qwen3-14b  29.00 GB  [bf16, awq-int4]
    a91c...  s3://acme-models/qwen3-14b-support@e3b0c442
 
$ npx @almyty/models register-version \
    --name qwen3-14b-support-ft \
    --uri s3://acme-models/qwen3-14b-support@e3b0c442
Registered.
qwen3-14b-support-ft  qwen3-14b  29.00 GB  [bf16, awq-int4]
    a91c...  s3://acme-models/qwen3-14b-support@e3b0c442

The URI must be pinned. An s3:// version reads its base, size and quantizations from almyty-manifest.json at the URI (the format is in the Models API). For every other scheme the manifest is optional; pass --base when it is absent, and --quantizations q1,q2 to record them.

Models: the list, and when a model is selectable
Your cloud account: each cloud, its sources, costs and what Stop means there
CLI overview: the other almyty CLIs and the shared login

Models CLI

Models

Your cloud account

Registered versions

Related