---
title: "Data and Sources"
url: /data
---

# Data and Sources

> How a product draws on what it did not build: Sources read live, mirrored or owned through Mappings, each value carrying who wrote it, when and its licence.

Data and Sources is how a product draws on what it did not build: each [Source](/patterns/dataset-api.md#how-a-field-is-filled)'s records become your Nouns through a [Mapping](/patterns/dataset-api.md#how-a-field-is-filled). Every value says who wrote it, when, and under what licence.

## Capability families
A [Dataset API](/patterns/dataset-api.md) resells facts, an [Agent API](/patterns.md#agent-api) works in each Customer's inbox and CRM, and a [Service API](/patterns.md#service-api) files with an outside office. Each draws on systems it does not control. It is not a proxy: a Source answers in its own shape, and the product answers in yours.

| Family | What it gives every product |
| --- | --- |
| Sources | What a product draws on: an origin, another [API product](/every-product.md), each Customer's own system, or people. Credentials are secrets, never in the file |
| resources and their modes | What a Source serves, by name, in one of three modes: upstream, asked on each call; mirrored, a kept copy; or owned, the product's own |
| Mappings | Each field read from a Source's field into yours, with a ranked `from` that tries each writer in turn, a [Function](/patterns.md#function-api) among them |
| [Provenance](/patterns/dataset-api.md#how-a-field-is-filled) | Every value carries its basis (Stated, Inferred or Observed), its writer, its time and its licence. Observed wins for facts |
| Freshness and coverage | A freshness target per resource, an age on every value, coverage per field, each measured from outside and printed on the offer card |
| Corrections | A Customer or a reviewer states a correction with its evidence. It is reviewed, applied, and sent upstream when the Source takes them |
| Licence terms | Redistribution, attribution and caching terms travel with every value, and the one decision refuses a use the licence forbids |
| [Integrations](/patterns/headless-saas.md#integrations) | Each Customer connects its own system once: tokens kept and refreshed, records read in and written back, the system's rate limits respected |
| Foreign truth | Stripe, Sources and connected systems are reconciled. What they report arrives as an Observed Event, and a conflict as an Event of its own |
| Outside submissions | A product files with an outside party and keeps its typed acknowledgement, such as a government office's receipt |
| Composition | Another API product as a Source, read through its Verbs at its printed price, pinned to its [Version](/reference/platform.md#version), its remedies flowing into yours |

The three modes are one choice per resource, and a field never knows which. The Dataset API's `firms` is mirrored, a Source with `live: true` is upstream, and a [Headless SaaS](/patterns/headless-saas.md)'s records are owned.

## Lifecycles and states
| Thing | States | What moves it |
| --- | --- | --- |
| A Source | `declared`, `probed`, `healthy`; or `failing`, `credential_expired` | The probe at deploy, each read, and a secret set again |
| An [Integration](/reference/platform.md#integration) | `offered`, `connecting`, `connected`, `syncing`; or `degraded`, `disconnected`, `revoked` | The Customer's consent, token refresh, the system's rate limits, the Customer's revoke |
| A mirrored value | `fresh`, `stale`, `refreshed`; or `superseded`, `corrected` | Its age past the target, a refresh, a higher-ranked writer, an accepted correction |
| A refresh | `scheduled`, `reading`, `written`; or `partial`, `retrying`, `failed` | The schedule's clock and the Source's answers; a written refresh writes `Company.refreshed` |
| A correction | `proposed`, `reviewed`, then `applied` or `declined`; `sent_upstream`, `acknowledged` | A reviewer's verdict, then the Source's answer |
| A submission | `submitted`, `acknowledged`, then `accepted` or `rejected` | The outside party's typed answer, arriving as an Event |

An Integration belongs to the Customer who connected it. The Customer revokes it, and its credentials never reach you.

## What you declare, what the platform does
| You declare | The platform does |
| --- | --- |
| A Source: its `origin`, `api` or `system`, and its `secret()` | Keeps the credential, probes the Source once at deploy, respects its rate limits, and retries its failures |
| Each resource's mode: `live`, or a `schedule` and a freshness target | Caches the upstream, refreshes the mirror, marks a stale value, and writes the `refreshed` Event |
| A Mapping per field, ranked when more than one writer can answer | Fills each field from the first writer that answers, stamps its Provenance, and keeps the others as history |
| A `license` | Prints it on the docs, carries it on every value, and refuses a use it forbids |
| `scope: 'customer'` on a Source | Serves the connect flow to each Customer as Verbs, keeps and refreshes their tokens, reads in and writes back |
| `cost` on a Source that is an API product | Prints your margin per Verb at deploy, pays the Source its printed price, and pins its Version |
| Nothing else | Reconciliation, conflict Events, typed acknowledgements, and the measurement of freshness and coverage |

### Example
A mirrored Source with a freshness target and a licence, and a ranked `from` with an agent behind it.

```ts title="definition.ts"
import { define, secret, agentic } from 'apis.dev'

/** Any company by its domain, with its headcount and sector. */
export default define({
  name: 'companyfacts', domain: 'companyfacts.apis.dev',
  …
  sources: {
    firms: {                                    // mirrored: a kept copy
      origin: 'https://firms.example.com', key: secret('FIRMS_KEY'),
      resources: { companies: { detail: '/companies/{website}' } },
      schedule: 'every Mon 06:00', fresh: '7d', license: 'CC-BY-4.0',
    },
  },
  Company: {
    domain:    { type: 'url!#',   from: 'firms', map: 'website' },
    employees: { type: 'number?', map: 'headcount',
                 from: ['firms', agentic('Find headcount for {domain}')] },  // ranked
    sector:    { type: 'string?', from: 'firms', map: 'industry' },
  },
  offer: { 'Company.get': 0.002, 'Company.search': 0.005 },
})
```

```json title="GET /events?type=Company.updated&after=evt_13"
{
  "items": [
    {
      "id": "evt_14",
      "type": "Company.updated",
      "at": "2026-10-26T06:00:00Z",
      "seq": 14,
      "via": "firms",
      "provenance": "Observed",
      "data": { "domain": "kestrel.example", "employees": 380 }
    }
  ],
  "next": null
}
```

Until `firms` covered `kestrel.example`, the agent's headcount answered, Inferred. Monday's refresh observed 380, and the change reached the feed and every Watch on `employees`.

## Contracts with the other subsystems
Foreign truth enters the product only as an Observed Event through the Runtime's gate.

| Subsystem | What Data and Sources takes from it | What it gives back |
| --- | --- | --- |
| Definition and compile | Sources, resources, Mappings, licences and costs, checked at deploy | An unknown Source or member refused; `below_cost` when a price is under a Source's cost; the probe's mapped record |
| Identity, access and policy | The Customer's consent to connect, held as a grant and revocable | Licence terms and consent, which the one decision reads before a value leaves |
| Model and records | The Nouns and fields a Mapping fills | Values stamped with their Provenance; every superseded value kept as history |
| Runtime | Steps, schedules, effects and the gate | Reads as steps (`AwaitingSource`), refreshes as scheduled [Triggers](/patterns.md#the-blocks), write-backs and submissions as effects |
| Work | Review of corrections, and human research as a writer | Outside submissions and their typed acknowledgements, for a case's steps |
| Commerce | Your prices, against which margin is computed | Each Source's cost per call; an API product's price, paid in the same settlement |
| Interfaces and distribution | `?include=provenance`, docs pages, the portal | Licence text per docs page; the connect flow as Verbs the portal renders; Watches on any field |
| Trust and operations | Secrets, brakes and the outside check | Freshness and coverage to measure; scraping against licence as a brake |

## What Customers and their agents meet
- **Your shape, never the Source's.** Customers call your Nouns and Verbs; the credential behind them never reaches them.
- **Where each value came from.** `?include=provenance` answers who wrote each value, on what basis, when, and under what licence.
- **Proof before the first call.** Freshness and coverage sit on the offer card, and a miss costs nothing.
- **Changes delivered.** A Watch on a field delivers each change a refresh or an Integration makes, in order.
- **Their own systems, connected headless.** A Customer connects its CRM or inbox from the portal or the API, and revokes it the same way.
- **Disagreements in the open.** When their system and the product disagree, an Event names both values. Observed wins for facts.
- **A correction is a Verb.** A Customer states the value with its evidence and watches the review decide it.

---

Every value carries its writer, its time and its licence, so outside truth is reconciled, never overwritten in silence.