Incrementality API

Overview

The Incrementality API lets an agency run geo-holdout tests on Meta and derive its own incrementality coefficient per channel — the iROAS ÷ platform_ROAS multiplier that converts Meta's reported ROAS into true incremental ROAS.

You define a test (which regions get ads, which are held out, and the before/after windows), we compute the lift entirely from data already in your BigQuery dataset, and the coefficient is served back as a precision-weighted running median over every test you've run — a "progressive truth" that keeps improving as you accumulate experiments.

All three inputs come from data you already have — no new connector, no manual data entry:

• Revenue by region — your resolved conversion events (which carry geo_region).
• Ad spend + platform-reported purchase value — your Airbyte-synced Meta Ads insights.

Base URL

All endpoints are mounted under the /client/v1/incrementality prefix.

https://mythic-analytics.gulp.workers.dev/client/v1/incrementality

Authentication

Same key system as the rest of the /client/v1 surface: an agency key (ak_, read + write, bind a location with ?location_id= or the X-Location-Id header) or a secret key (sk_, read-only viewer, auto-scoped to its location). Creating and computing experiments requires an ak_ key; reads accept either.

Quickstart — from "should we?" to a coefficient

# easiest: let the API find and assess every conversion event for you
curl "$BASE/eligibility?auto=true" -H "Authorization: Bearer $AK"

# or assess one revenue source explicitly:
curl "$BASE/eligibility?daily_budget=500" -H "Authorization: Bearer $AK"
# lead-gen client? assess a geo'd browser conversion instead:
curl "$BASE/eligibility?event_name=form_submitted&measure=count" -H "Authorization: Bearer $AK"

curl "$BASE/markets?test_regions=California,Texas" -H "Authorization: Bearer $AK"

curl -X POST "$BASE/experiments" -H "Authorization: Bearer $AK" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Q3 Meta geo holdout",
    "definition": {
      "test_regions": ["California", "Texas"],
      "control_regions": ["Ohio", "Georgia"],
      "test_window": { "from": "2026-06-01", "to": "2026-06-30" },
      "daily_budget": 500
    }
  }'

curl "$BASE/experiments/{id}/launch-plan" -H "Authorization: Bearer $AK"

"progress": {
  "phase": "running",
  "days_total": 30, "test_days": 30, "cooldown_days": 0,
  "days_elapsed": 12, "days_remaining": 18,
  "percent": 40, "recommended_days": 28, "underpowered": false
}

curl -X POST "$BASE/experiments/{id}/compute" -H "Authorization: Bearer $AK"

curl "$BASE/coefficients?channel=meta" -H "Authorization: Bearer $AK"

Statistical model

Inference is weighted synthetic control difference-in-differences: a non-negative weighted blend of the control regions is fit on the pre period to reproduce the test regions' daily revenue (weights and fit quality are reported as control_weights / fit_r2), the daily residual test − blend gives the noise floor, and the mean-adjusted post-period residual sum is the incremental revenue. Its standard error comes from pre-period residual variance, inflated by a degrees-of-freedom correction for the fitted weights. Significance is one-sided at 95% ("did the channel drive anything?") and the reported intervals are 90% two-sided, so the verdict and the interval never disagree. The create-time power analysis solves the same model forward: recommended_days = ((z_α + z_power) · σ / (daily_budget × mde_iroas))², with mde_iroas defaulting to 1.0 — "detect it if the channel at least breaks even."

Every calendar day in the measured windows counts: a day with zero conversions is a zero, not a missing observation.

Validation, seasonality, and data quality

• A/A placebo before launch — GET /experiments/{id}/validate (while the test hasn't started) splits the pre period in half and runs the exact inference with the second half as a fake "post". True lift is zero by construction, so a placebo lift significant in either direction (aa_pass: false, |z| ≥ 1.96) means the market pairing is invalid — fix the regions before spending a dollar. market_match_quality is the synthetic-control fit R².
• Drift monitoring while running — the same endpoint in a running test returns interim lift (labeled peeking: true — not a result), and drift_detected when outlier days appear inside the test window. (control_shift_z is reported but informational: control regions are held out, so their level shift vs pre is the intended treatment contrast — not drift.) Set definition.alert_webhook_url (https) and Mythic checks daily through the test and cooldown, POSTing an incrementality.drift_alert webhook only when the outlier set changes — alert state is persisted, so the same condition never alerts twice. Webhook requests carry X-Mythic-Event: incrementality.drift_alert and X-Mythic-Experiment-Id headers.
• Seasonality — matched controls absorb shared seasonality by construction (both groups live through the same holidays and promos). What they don't absorb is guarded: non-whole-week windows bias the fit via day-of-week composition (windows_not_whole_weeks warning — size windows in multiples of 7), and region-specific drift is what the weighted blend minimizes.
• Outliers — days whose residual is >3σ off the pre-period distribution are reported in outlier_days, never silently removed. Exclude them explicitly with definition.exclude_dates.
• Conversion lag — definition.cooldown_days extends the measured window past the test window so lagged conversions land inside the measurement; the launch plan keeps geo exclusions in place through the cooldown.

How the coefficient is computed

Two queries against your per-location BigQuery dataset — daily revenue-by-region, and the Meta spend row — with inference in between:

1. Build daily revenue series for the test and control regions across the pre and test windows (every calendar day counts, zeros included).
2. Fit the counterfactual: a non-negative weighted blend of the control regions that reproduces the test regions' pre-period daily revenue (weighted synthetic control).
3. incremental_revenue = the mean-adjusted sum of post-period residuals test − blend.
4. iROAS = incremental_revenue ÷ test_spend, where test spend comes from Meta.
5. coefficient = iROAS ÷ platform_ROAS (equivalently incremental_revenue ÷ platform_reported_revenue).

A coefficient of 1.0 means Meta's reported ROAS is exactly right; 1.15 means Meta under-reports; 0.25 means three-quarters of that reported ROAS was not incremental.

Defining an experiment

The definition object on an experiment carries the full test spec:

Revenue source (revenue)

Configurable per experiment, because revenue lives in different places for different clients:

• Event JSON (default) — { "source": "event", "event_name": "purchase", "json_path": "$.value" }. Reads revenue from the tracked conversion event; region comes from the event's geo. properties_json IS the event's properties object, so the default path is $.value. Add "timezone": "America/New_York" to align the revenue day boundary with the ad account (default UTC).

  The default path is right for SDK capture() revenue and wrong for most CRM streams. GoHighLevel transaction_completed events, for example, carry the amount at $.meta_amount and are 0% populated at $.value.

  A path that extracts nothing does not error — every revenue figure simply becomes 0, which also drives revenue-weighted geo_coverage to 0. The eligibility check now separates the two cases: when matching events exist and carry a region but no number at the configured path, root_cause.check is revenue_value (fix the path) rather than geo_coverage (fix tracking). Run GET /incrementality/eligibility?auto=1 to see the path each event actually uses.

• Lead-gen count mode — { "source": "event", "event_name": "form_submitted", "measure": "count" }. Counts matching events instead of summing a value, for conversion events that carry no monetary amount (form fills, lead captures). iroas then reads as incremental conversions per dollar; pair with "spend": { "purchase_action_type": "lead" } so the platform side reads Meta's lead metric. Count-mode and revenue-mode experiments pool in separate coefficient series.

  Count-mode results are self-describing so the units cannot be misread: measure: "count", lift_units: "conversions", iroas_units, and cost_per_incremental_conversion (the number most readers actually want). coefficient is always null in count mode — it is revenue-denominated by construction, so a conversion count cannot produce one. coefficient_unavailable_reason says so on the response. If you need the coefficient, you need measure: "revenue" with a populated json_path.

• Commerce table — { "source": "table", "table": "transactions", "column": "amount", "region_col": "state", "date_col": "created_date" }. Uses a structured table; that table must carry its own region and date columns to be geo-splittable.

Spend attribution (spend)

• Geo stream (preferred) — { "source": "geo_stream", "region_col": "region" }. Uses the region-broken Meta insights stream (meta_ads_ads_insights_region), so test-region spend is read directly from Meta. Set "table" only if your geo-broken insights live in a custom table.

  The region stream is a prerequisite, and older connections do not have it. It is a default on Meta connections created by the platform today, but a connection created before the stream was added does not gain it retroactively — enabling a stream on a live connection is an explicit action, because it triggers a full backfill from the source's start_date. On a two-year history that first sync can take many hours.

  If spend.source is geo_stream and the table is absent you get 422 spend_table_missing. Enable ads_insights_region on the connection (or use source: "campaigns" meanwhile) and wait for the backfill to finish before trusting a result — a partially-synced region table yields spend that is real but incomplete, which is worse than a missing one because it still produces a number.

• Campaigns — { "source": "campaigns", "campaign_ids": ["123"] } or { "source": "campaigns", "name_pattern": "GEOTEST" } (a literal substring — %/_ are not wildcards). Attributes spend to the test by which campaigns target the test regions, for accounts without the geo stream. Add "account_id": "act_…" when the dataset receives more than one Meta connection.

With no geo/campaign filter, spend falls back to whole-account — which is only meaningful because the holdout excludes control regions (account spend ≈ test spend). Always scope spend explicitly for partial holdouts or multi-account datasets.

Platform-reported conversion value is read from Meta's action_values using one canonical action type — omni_purchase, falling back to purchase (Meta reports the same conversion under several overlapping types, so they are never summed together). Lead-gen accounts can override with "purchase_action_type": "lead" (or any Meta action type) on the spend object. The default table is meta_ads_ads_insights_action_type, also synced by default on platform-created Meta connections.

The result

After POST /experiments/{id}/compute, the experiment's result holds the computed metrics: incremental_revenue, test_spend, platform_revenue, iroas, platform_roas, coefficient, the absolute standard errors (se_lift, se_iroas, se_coefficient), ci_width (relative 90% CI half-width, display only), series_key, spend_currency, spend_data_through, plus the raw test_pre/test_post/control_pre/control_post (note: the control totals are the weighted blend, scaled to match test) and computed_at.

GET /coefficients returns the inverse-variance weighted median coefficient per series_key across computed experiments — weight = 1/se_coefficient², so precision (not effect size) determines pull, and a huge noisy lift earns no extra say. Results computed mid-window (partial) or with a poor control fit (poor_control_fit warning) are excluded.

Response envelope

All endpoints return a JSON envelope.

Error handling