---
title: "Alerts and on-call"
url: /trust/alerts
---

# Alerts and on-call

> Alert rules on errors, time, refusals, queues, Sources and spend, routed to the Role on call and handed on when nobody acknowledges.

An alert rule watches one measure of the running product, and its route reaches the person on call, then the next, until someone acknowledges.

```mermaid title="An alert, from a measure to the person on call"
stateDiagram-v2
  [*] --> armed: a rule deploys
  armed --> pending: over the line
  pending --> armed: back under
  pending --> firing: held for
  firing --> handed_on: nobody
  firing --> acknowledged: on call
  handed_on --> acknowledged: next route
  acknowledged --> resolved: under again
```

## One night on call
Follow one rule from a quiet product to a resolved alert.

The rule pages the admin on call when a Verb's errors pass 2% for 5 minutes.

1. **02:10.** Errors on the Verb reach 3.4%. The rule moves from `armed` to `pending`; nobody is told yet.
2. **02:15.** The share has held past the line for 5 minutes, the rule's `for`. It fires: one page and one chat message to the admin on call, with the reading, the rule and the call that crossed the line.
3. **02:30.** Nobody has acknowledged within the route's `after`. The alert is handed on to the route `next` in line, with everything the first route sent.
4. **02:33.** The owner taps acknowledge. The hand-on stops everywhere on the line.
5. **02:51.** Errors fall back under the line. The alert resolves itself and records how long it held.

However many calls cross the line, a rule fires once while it holds. A rule whose measure falls back before `for` passes returns to `armed` without a sound. Customers see nothing, unless the alert becomes an incident.

## What a rule can watch
Every measure comes from the [Call record](/reference/platform.md#call-record) and the trace, so an alert and the call that caused it share one id.

| Measure | What it reads |
| --- | --- |
| `errors` | The share of a Verb's calls that fail |
| `median`, `slowest` | How long a Verb's calls take |
| `refusals` | Each refusal callers met, by code |
| `queue` | Tasks waiting for a Role |
| `source` | Failures reading a [Source](/patterns/dataset-api.md#how-a-field-is-filled) |
| `spend` | Spend on models and Sources |
| `disputes` | Disputes opened |

A rule in `operate.alerts` names a measure and the Verb, Role or Source it reads. It names the line it crosses (`above`) and how long it holds there (`for`). A route in `operate.routes` names its channels, the Role of your team on call, and the route `next` in line `after` a wait.

## Routes, silences and the tools you already run
Rules live in the file beside the promise they protect, and are reviewed on a [branch](/definition/changes.md) like any change.

### Example
The [Function](/patterns/function-api.md) API pages the admin on call when summaries fail or research slows, and tells the team when the human queue grows.

```ts title="definition.ts"
import { define, generative, agentic, human } from 'apis.dev'

/** Answers about any company by its domain: a summary in seconds, research in minutes, a person's check in hours. */
export default define({
  name: 'summaries', domain: 'summaries.apis.dev',
  Company: {
    domain:    'url!#',
    summarize: generative('Two sentences on what {domain} sells', { returns: { summary: 'string' } }),
    research:  agentic('Find the head of sales at {domain}', { returns: { headOfSales: 'string' } }),
    verify:    human('Call and confirm the head of sales',
      { role: 'researcher', within: '24h', returns: { confirmed: 'boolean' } }),
  },
  offer: { 'Company.summarize': 0.02, 'Company.research': 0.5, 'Company.verify': 5 },
  operate: {
    alerts: {
      failing: { metric: 'errors', verb: 'Company.summarize', above: 0.02, for: '5m', to: 'oncall' },
      slow:    { metric: 'median', verb: 'Company.research', above: '3m', for: '15m', to: 'oncall' },
      backlog: { metric: 'queue', role: 'researcher', above: 25, for: '30m', to: 'team' },
    },
    routes: {
      oncall: { send: ['page', 'chat'], role: 'admin', next: 'lead', after: '15m' },
      lead:   { send: ['page'], role: 'owner' },
      team:   { send: ['chat', 'email'] },
    },
  },
})
```

Your operator agent lists what is firing, with the call that crossed the line.

```http title="Response"
HTTP/1.1 200 OK
Content-Type: application/json
X-Cost: price=0, currency="USD"
X-Request-Id: req_58c2e0f7a19d4b3e6c71

{
  "items": [
    {
      "id": "alert_31",
      "rule": "failing",
      "state": "firing",
      "reading": { "errors": 0.034, "above": 0.02, "for": "7m" },
      "route": "oncall",
      "sentTo": ["page", "chat"],
      "handOnAt": "2026-10-27T15:22:00Z",
      "example": "req_a41f9c07e2b85d3c6f10"
    }
  ],
  "total": 1,
  "next": null
}
```

The list is `GET https://apis.dev/v1/products/summaries/alerts`, `npx apis.dev alerts list`, or `listAlert` over MCP.

### Silences, incidents and your own tools
Email routes work on every plan; chat, paging and webhook routes arrive with Team.

[/pricing](/pricing.md) prints the channels per plan. With no rules at all, the workspace owner gets an email for errors above 5% for 5 minutes and for a Source down for 15.

- **Silences.** `Alert.silence` holds a rule quiet for a window, with a reason kept beside it; the deploy that changes the rule ends the silence.
- **Incidents.** An alert on a measure Customers feel opens a draft [incident](/trust.md) on the status page, which your operator confirms or discards.
- **Your operator agent.** `Alert.list` and `Alert.acknowledge` under its token, with the trace behind each reading and the [rollout](/definition/rollout.md) or deploy that came before it.
- **Your own tools.** Every alert is an Event, so a [sink](/model/analytics.md#reading) carries it to the paging or observability tool you already run.
- **Customers' own alerts.** A Customer watches its own usage and spend with [Watches](/reference/platform.md#watch), never with your rules.

An alert is `firing`, `acknowledged`, then `resolved`; each hand-on is `sent`, `handed_on` or `acknowledged`. A rule is `armed`, `pending`, `firing` or `silenced`.