Price and offer variants
Open in
A variant is another version of the Offer, shown to a share of new Customers and measured against the Offer as written.
Filled: a state someone outside the run sees. Hollow: a private state, a step inside the run.
What to test, and how to write it
Start from the question you want answered. Each row is one variant in offer.variants, beside the control.
| You want to know | The variant changes | The goal to measure |
|---|---|---|
| Whether a higher price loses buyers | A plan's price | revenue, net revenue per exposure in the window |
| Whether a trial brings more paying Customers | A plan's trial | paid, a first paid charge |
| Whether more included calls keep Customers | A plan's includes or seats | retained, still paying after 30 days |
| Whether a lower price per call grows use | A price per call, by consumption or by outcome | revenue |
| Whether a free plan brings sign-ups | Which plans the Offer holds | signup |
A variant never changes the terms, the refund rule or the licence. The stop rule is experiment.exposures per variant or within, whichever comes first, and an experiment never concludes on less.
Who is in it, and what each sees
Visitors, new sign-ups and buyer accounts with no plan yet join; a current Customer and a signed agreement keep their prices.
| Who | What they see |
|---|---|
| A visitor | One variant, assigned at its first exposure and kept for good across devices and every Interface; the same price after sign-up, and a line on the offer card that a test is running |
| A Customer already paying | The price it signed up at, until a change reaches it by the notice and grandfathering rules on agreements and price changes |
| A buyer's agent | One variant per buyer account on every card it fetches; the signed card, the quote and the receipt carry it, so it pays the price it compared |
Anyone reading /pricing.json | The variant its caller sees, named |
| You | The analysis per variant, the interval for each difference, and the one Verb that keeps a variant |
For one Customer, /pricing, X-Cost, the offer card, the quote and the receipt print the same assigned price. An exposure is the first time a variant's price is shown there, or on a 402 offer or Checkout. Each is written as an Event.
Reading the result
The analysis is a projection of the Event stream, so it and the bill read the same records.
Example
The weekly brief tests a higher Member price and a free trial, each on a quarter of new readers.
import { define, generative } from 'apis.dev'
/** A weekly brief on the software companies that changed headcount, for members and their agents. */
export default define({
name: 'briefs', domain: 'briefs.apis.dev',
sources: {
firms: { api: 'companyfacts.apis.dev', cost: { 'Company.search': 0.005 } },
},
Issue: {
week: 'string!#', title: 'string', body: 'markdown',
status: {
draft: { run: generative('This week in {firms}: who hired, who shrank') },
review: { in: 'drafted', role: 'editor' },
publish: { in: 'reviewed', role: 'editor' },
send: { in: 'published', to: 'Member' },
},
},
on: { 'every Mon 06: 00': 'Issue.draft', 'Issue.published': 'Issue.send' },
offer: {
'Issue.list': 0, 'Issue.read': 0.25,
Member: { month: 8, includes: { 'Issue.read': 'all' } },
variants: {
higher: { share: 0.25, Member: { month: 10, includes: { 'Issue.read': 'all' } } },
trial: { share: 0.25, Member: { month: 8, trial: '14d', includes: { 'Issue.read': 'all' } } },
},
experiment: { goal: 'revenue', exposures: 1000, within: '30d' }, // half see the Offer as written
},
})After 30 days, the experiment is ready, and neither variant is clear of the control.
HTTP/1.1 200 OK
Content-Type: application/ json
X-Cost: price= 0, currency= "USD"
X-Request-Id: req_b2d07e45c9a1f3860d4e
{
"id": "exp_3",
"state": "ready",
"goal": "revenue",
"variants": [
{ "name": "control", "exposed": 2140, "converted": 96, "perExposure": 0.36 },
{ "name": "higher", "exposed": 1066, "converted": 41, "perExposure": 0.38,
"difference": { "low": -0.03, "high": 0.08 } },
{ "name": "trial", "exposed": 1072, "converted": 52, "perExposure": 0.39,
"difference": { "low": -0.02, "high": 0.08 } }
],
"clear": false
}Read it with GET https:, npx apis.dev experiments get, or getExperiment over MCP.
Each variant prints its exposures, its conversions, the rate, revenue per exposure, and a 95% interval for its difference from the control. Here both intervals cross zero, so clear is false and neither variant has earned a decision.
Keeping a variant
Deciding what to test and which variant to keep stays yours.
Experiment.conclude keeps one, and the next deploy writes it as the Offer; Customers assigned another keep their price by changes.grandfather.
The experiment is running, ready, then concluded or stopped; a deploy without variants stops it. A variant runs until the conclusion makes it kept or retired. An assignment is assigned, exposed, then converted when the goal is met inside the window, or lapsed.
Plan changes and proration
What a Customer pays when its plan or seats change inside a period: up at once and prorated, down at renewal, previewed to the cent first.
Referral codes and affiliates
Credit for a Customer who brings another, and cash for an affiliate who brings many: codes, attribution, holds against refunds, payouts and abuse checks.