---
title: "Canaries and gradual release"
url: /definition/rollout
---

# Canaries and gradual release

> A deploy reaches Customers in steps: a canary first, each step held and guarded, and an automatic rollback when the new Version does worse.

Every deploy to Live reaches Customers in steps: a canary first, then larger shares, each held while guards compare it with the [Version](/reference/platform.md#version) before.

```mermaid title="A rollout: its states"
stateDiagram-v2
  [*] --> canary: a deploy to Live
  canary --> stepping: guards hold
  stepping --> promoted: last step
  stepping --> rolled_back: fails
```

Seen: promoted, rolled_back. Private, a step inside the run: canary, stepping.

## One deploy, hour by hour
A rollout moves Customer orgs, never single calls. Here is one, with a canary at 5%, steps held two hours, and the guards it carries.

| Hour | On the new Version | What happens |
| --- | --- | --- |
| 0 | 5% of orgs, picked at random | The [Verifier](/trust/verifier.md)'s suite has passed; the canary starts, and no org pinned to a Version is picked |
| 0 to 2 | 5% | Every minute, the guards compare the new Version with the old over the same minutes |
| 2 | 25% | The hold passed with every guard holding, so the next step takes its share |
| 4 | 50% | The same again |
| 6 | Every org | The last step: the rollout is `promoted`, and the Version before is `superseded` |
| Any minute | Back to none | One guard fails: every org on the new Version moves back within a minute, and the rollout ends `rolled_back` |

During a rollout, an org's API, MCP, SDK, site, portal and Watches all answer from one Version. Records written under the new Version stay after a rollback; the Version before reads them, because a rollout only carries changes it can read.

A rollout is `canary`, `stepping` or `paused`, then `promoted` or `rolled_back`. Each Customer org in it is `old`, then `new`, and either stays `new` for good or is `moved_back`. A guard is `holding` until one comparison leaves it `failed`.

## The guards
Each guard reads one measure of the new Version against the old, and one failure ends the rollout.

| Guard | What it reads | It fails when |
| --- | --- | --- |
| `errors` | The share of calls that fail on the platform's side or the [Function](/patterns/function-api.md)'s, never a refusal a caller earned | The new share is above the old by more than `errors` |
| `slower` | The median time of each Verb | The new median is slower than the old by more than that share |
| `evals` | Outputs of each generative and agentic Verb, sampled and scored by its evals | The new score falls below `evals` |
| Conformance | The [Verifier](/trust/verifier.md)'s suite, run on the new Version before the canary | Any check fails; nothing is declared |

Every step and every failure writes an Event, so a [Watch](/reference/platform.md#watch) or an [alert route](/trust/alerts.md) hears it.

## What rolls out, and what never does
Only Live rolls out. Find the change you are shipping.

| The change | How it reaches Customers |
| --- | --- |
| A new prompt, a price, a new Verb or field | In steps, guarded, as above |
| A removal, or a narrowed type | Never in steps: deploy refuses it with `unsafe_rollout`, and the change ships in two deploys |
| A breaking change | As a new Version each [Key](/reference/platform.md#key) moves to by sending `Api-Version`; the rollout governs which new Keys start on it |
| Anything on Test or a [branch](/definition/changes.md) | At once |
| A rollout you want to hold, finish or undo | Build plane Verbs: `Rollout.pause` holds it at a step, `Rollout.promote` takes it to everyone with a person's approval, and `Rollout.stop` sends everyone back |

### Example
The agent product changes its booking prompt, and the change reaches 5% of Customers first.

```ts title="definition.ts"
import { define, agentic } from 'apis.dev'

/** An agent that books three meetings a week with each Customer's leads, in their own inbox and CRM. */
export default define({
  name: 'meetings', domain: 'meetings.apis.dev',
  sources: {
    inbox: { system: 'email', scope: 'customer' },
    crm:   { system: 'crm',   scope: 'customer' },
  },
  Meeting: {
    with: 'string', company: 'string', at: 'date',
    book: agentic("Book a meeting with the next lead in {crm}, in the lead's own time zone",
      { tools: ['inbox', 'crm'], returns: { meeting: 'string' } }),
  },
  on: { 'every Mon, Wed, Fri 09:00': 'Meeting.book' },
  offer: {
    'Meeting.book': { per1000Tokens: 0.01, perMinute: 0.05 },
    'Meeting.get': 0, 'Meeting.list': 0,
  },
  operate: {
    promise: { 'Meeting.book': { success: 0.99, median: '10m' } },
    rollout: {
      canary: 0.05, steps: [0.25, 0.5, 1], hold: '2h',    // each step held two hours
      guard: { errors: 0.01, slower: 0.25, evals: 0.9 },   // past any, every Customer goes back
    },
  },
})
```

Your coding agent reads the rollout between steps.

```http title="Response"
HTTP/1.1 200 OK
Content-Type: application/json
X-Cost: price=0, currency="USD"
X-Request-Id: req_4e81c0a2d9b7f3561e0c

{
  "id": "ro_7",
  "state": "stepping",
  "version": "v4",
  "from": "v3",
  "step": { "share": 0.25, "next": "2026-10-27T16:00:00Z" },
  "customers": { "v4": 31, "v3": 89, "pinned": 4 },
  "guards": {
    "errors": { "new": 0.004, "old": 0.003, "limit": 0.01, "ok": true },
    "slower": { "new": "6m", "old": "6m", "limit": 0.25, "ok": true },
    "evals": { "new": 0.93, "limit": 0.9, "ok": true }
  }
}
```

Your agent sent `GET https://apis.dev/v1/products/meetings/rollouts/ro_7`; `npx apis.dev rollouts get` and `getRollout` over MCP answer the same.