Analytics

Analytics shows what your assistants and tools have been doing and what it cost. You can see how many questions the support assistant answered today, which tools it used, which runs failed, and what each model has cost this month. Every sensitive change anyone made is recorded here too, so you can answer “who changed this agent, and when?”

Find your way around

1. Open Analytics in the sidebar. The Overview tab opens first.
2. Pick a tab to look at one area in detail. Each tab has its own address, so you can send a colleague a link to exactly the view you are looking at.
3. On Tools, Gateways and Models, choose the time range at the top. Agents always shows the last 24 hours and the last 7 days, and Cost switches between today and this month.
4. To download the list of incoming requests, click Export CSV or Export JSON at the top of the page.

What each tab shows

Overview. The headline numbers for the last 24 hours (requests, tool calls, average response time, errors, model calls) and the last 7 days (requests, tool calls, model cost), with a chart of request volume over the week. It refreshes itself every 30 seconds.

Request log. Every request that came in through a gateway (the address your tools are published on, see Gateways), with its time, status and how long it took. Filter it to only the successes or only the errors.

Tools. For each tool: how often it ran, how often it worked, how long it took and when it was last used. It counts every way a tool runs: a test from the tool’s page, a call from an outside AI app, or a step in one of your agents.

Gateways. For each gateway: requests, successes, errors and success rate. A gateway with lots of errors is usually set up wrong; one with no traffic may not be needed.

Models. Usage per model provider: conversations, messages, text in and out (counted in tokens, the unit model vendors bill by), tool calls and cost.

Routing. For agents that can fall back from one model to another, how often every model they tried failed. Next to it, as a separate number, how often a better choice of models could have done better. An agent needs a minimum number of requests where more than one model was tried before a rate is shown; agents below that are listed in their own card, so “working fine” and “not measured yet” never look the same.

Agents. For each agent: status, total runs, runs in the last 24 hours and 7 days, success rate, average time and cost, with the most recent failures underneath.

Cost. What agent runs cost, today or this month: the total, spend over time and spend per agent. Below it, Provider actual vs our estimate compares almyty’s own estimate with what your model vendor says you spent, and Sync now reads the vendor’s numbers again. See Cost governance.

Budgets. Spending limits for the whole organization or for one agent, with what has been spent so far this period and a list of Recent budget alerts. Admins and owners can add and change budgets; everyone else can look. See Cost governance.

Chargeback. What each team and each agent spent this period, their share of the total, and a projection for the rest of the period (a trend, not a promise). This is an Enterprise plan feature; without it the tab shows an upgrade prompt.

Audit trail. The audit log: a record of every sensitive action, with when, who, what they did, to which item, whether it worked and from which IP address. Filter it by the kind of item and the kind of action. Everyone can read it; downloading it as a file needs the Business or Enterprise plan (see Audit export).

How long data is kept

By default nothing is deleted. To keep data for a limited time, open Settings → Organization → Details and use the Data retention card. Set a number of days (1 to 3650) for each kind of data: agent runs, conversations, request logs, usage metrics, the audit log, tool executions and notifications. Leave a field empty to keep that kind forever. A clean-up runs every hour and removes whatever is older than you allowed.

For developers

GET /analytics/overview returns what the Overview tiles show, in two windows:

The per-item reports (/analytics/tool-usage, /analytics/gateway-usage, /analytics/llm-usage, /analytics/agent-runs) each carry their own counts and a successRate over the timeframe you ask for. The Tools report counts direct test runs, MCP tools/call, UTCP execute and agent tool_call steps alike.

Tab addresses follow /analytics/<tab>, for example /analytics/tools and /analytics/budgets.

Retention can also be set over REST with PUT /organizations/:id/retention. Each window is 1 to 3650 days, and an empty one keeps that class forever.

Usage metrics, request logs, audit logs and CSV or JSON export are all in the API reference.