Skip to content

Alerts and on-call

An alert rule watches one measure of the running product, and its route reaches the person on call, then the next, until someone acknowledges.

An alert, from a measure to the person on call

One night on call

Follow one rule from a quiet product to a resolved alert.

The rule pages the admin on call when a Verb's errors pass 2% for 5 minutes.

  1. 02:10. Errors on the Verb reach 3.4%. The rule moves from armed to pending; nobody is told yet.
  2. 02:15. The share has held past the line for 5 minutes, the rule's for. It fires: one page and one chat message to the admin on call, with the reading, the rule and the call that crossed the line.
  3. 02:30. Nobody has acknowledged within the route's after. The alert is handed on to the route next in line, with everything the first route sent.
  4. 02:33. The owner taps acknowledge. The hand-on stops everywhere on the line.
  5. 02:51. Errors fall back under the line. The alert resolves itself and records how long it held.

However many calls cross the line, a rule fires once while it holds. A rule whose measure falls back before for passes returns to armed without a sound. Customers see nothing, unless the alert becomes an incident.

What a rule can watch

Every measure comes from the Call record and the trace, so an alert and the call that caused it share one id.

MeasureWhat it reads
errorsThe share of a Verb's calls that fail
median, slowestHow long a Verb's calls take
refusalsEach refusal callers met, by code
queueTasks waiting for a Role
sourceFailures reading a Source
spendSpend on models and Sources
disputesDisputes opened

A rule in operate.alerts names a measure and the Verb, Role or Source it reads. It names the line it crosses (above) and how long it holds there (for). A route in operate.routes names its channels, the Role of your team on call, and the route next in line after a wait.

Routes, silences and the tools you already run

Rules live in the file beside the promise they protect, and are reviewed on a branch like any change.

Example

The Function API pages the admin on call when summaries fail or research slows, and tells the team when the human queue grows.

definition.ts
26 lines
import { define, generative, agentic, human } from 'apis.dev'

/** Answers about any company by its domain: a summary in seconds, research in minutes, a person's check in hours. */
export default define({
  name: 'summaries', domain: 'summaries.apis.dev',
  Company: {
    domain:    'url!#',
    summarize: generative('Two sentences on what {domain} sells', { returns: { summary: 'string' } }),
    research:  agentic('Find the head of sales at {domain}', { returns: { headOfSales: 'string' } }),
    verify:    human('Call and confirm the head of sales',
      { role: 'researcher', within: '24h', returns: { confirmed: 'boolean' } }),
  },
  offer: { 'Company.summarize': 0.02, 'Company.research': 0.5, 'Company.verify': 5 },
  operate: {
    alerts: {
      failing: { metric: 'errors', verb: 'Company.summarize', above: 0.02, for: '5m', to: 'oncall' },
      slow:    { metric: 'median', verb: 'Company.research', above: '3m', for: '15m', to: 'oncall' },
      backlog: { metric: 'queue', role: 'researcher', above: 25, for: '30m', to: 'team' },
    },
    routes: {
      oncall: { send: ['page', 'chat'], role: 'admin', next: 'lead', after: '15m' },
      lead:   { send: ['page'], role: 'owner' },
      team:   { send: ['chat', 'email'] },
    },
  },
})

Your operator agent lists what is firing, with the call that crossed the line.

Response
HTTP/1.1 200 OK
Content-Type: application/json
X-Cost: price=0, currency="USD"
X-Request-Id: req_58c2e0f7a19d4b3e6c71

{
  "items": [
    {
      "id": "alert_31",
      "rule": "failing",
      "state": "firing",
      "reading": { "errors": 0.034, "above": 0.02, "for": "7m" },
      "route": "oncall",
      "sentTo": ["page", "chat"],
      "handOnAt": "2026-10-27T15:22:00Z",
      "example": "req_a41f9c07e2b85d3c6f10"
    }
  ],
  "total": 1,
  "next": null
}

The list is GET https://apis.dev/v1/products/summaries/alerts, npx apis.dev alerts list, or listAlert over MCP.

Silences, incidents and your own tools

Email routes work on every plan; chat, paging and webhook routes arrive with Team.

/pricing prints the channels per plan. With no rules at all, the workspace owner gets an email for errors above 5% for 5 minutes and for a Source down for 15.

  • Silences. Alert.silence holds a rule quiet for a window, with a reason kept beside it; the deploy that changes the rule ends the silence.
  • Incidents. An alert on a measure Customers feel opens a draft incident on the status page, which your operator confirms or discards.
  • Your operator agent. Alert.list and Alert.acknowledge under its token, with the trace behind each reading and the rollout or deploy that came before it.
  • Your own tools. Every alert is an Event, so a sink carries it to the paging or observability tool you already run.
  • Customers' own alerts. A Customer watches its own usage and spend with Watches, never with your rules.

An alert is firing, acknowledged, then resolved; each hand-on is sent, handed_on or acknowledged. A rule is armed, pending, firing or silenced.