Provision workspace
Point the agency at an Airbyte workspace and a BigQuery destination, either by CREATING them or by ADOPTING ones it already runs. Requires an agency key (ak_).
What repeats safely. The workspace is reused, never duplicated. A destination Mythic created is UPDATED IN PLACE, so re-running provision to change dataset, location or loading method repoints the same destination instead of leaving the previous one orphaned in Airbyte. A destination you ADOPTED is never modified or deleted — it is yours, and may serve your own pipelines; a later create-mode provision builds a separate Mythic-owned destination rather than rewriting yours. The response reports destination_reused so you can tell which happened.
Connections are unaffected either way: they reference the destination by id, and updating in place keeps that id.
Adopt what you already run. If the agency already syncs these platforms with its own Airbyte, pass airbyte_workspace_id + airbyte_destination_id (plus airbyte_client_id/ airbyte_client_secret when that setup is in the agency's own Airbyte org). Provisioning a second, independent workspace against the same BigQuery dataset produces duplicate rows, two sets of incremental cursor state that each believe they own the stream, and two Airbyte bills — and nothing errors. In CREATE mode the target dataset is checked for existing Airbyte-managed tables and the request is refused with 409 dataset_in_use unless confirm_shared_dataset: true.
Staging is optional, but all-or-nothing. Supply staging_bucket_name + hmac_key_access_id + hmac_key_secret together for GCS staging, or none of them for standard inserts. A partial set is rejected with 400 staging_incomplete.
Nothing is persisted by a failed provision that had no prior config: the row is rolled back, so GET /config keeps returning 404 not_configured rather than a permanently empty status: error config you cannot tell apart from a never-provisioned agency. Failures carry the underlying reason in the response — you should never have to read it back off GET /config.
curl -X POST "https://mythic-analytics.gulp.workers.dev/client/v1/airbyte/config/provision" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_API_TOKEN" \
-H "Authorization: Bearer YOUR_API_TOKEN" \
-d '{
"airbyte_workspace_id": "4fed5b02-4acd-4111-95bc-f7d15153b362",
"airbyte_destination_id": "613f34e2-bcc0-4bc5-ab3f-eeeff4647401",
"airbyte_client_id": "example_string",
"airbyte_client_secret": "example_string",
"confirm_shared_dataset": true,
"bigquery_project_id": "acme-analytics",
"bigquery_dataset": "mythic_ads",
"gcp_credentials_json": "example_string",
"dataset_location": "US",
"staging_bucket_name": "acme-airbyte-staging",
"staging_bucket_path": "example_string",
"hmac_key_access_id": "example_string",
"hmac_key_secret": "example_string",
"keep_files_in_gcs_bucket": "example_string"
}'
import requests
import json
url = "https://mythic-analytics.gulp.workers.dev/client/v1/airbyte/config/provision"
headers = {
"Content-Type": "application/json",
"Authorization": "Bearer YOUR_API_TOKEN"
}
data = {
"airbyte_workspace_id": "4fed5b02-4acd-4111-95bc-f7d15153b362",
"airbyte_destination_id": "613f34e2-bcc0-4bc5-ab3f-eeeff4647401",
"airbyte_client_id": "example_string",
"airbyte_client_secret": "example_string",
"confirm_shared_dataset": true,
"bigquery_project_id": "acme-analytics",
"bigquery_dataset": "mythic_ads",
"gcp_credentials_json": "example_string",
"dataset_location": "US",
"staging_bucket_name": "acme-airbyte-staging",
"staging_bucket_path": "example_string",
"hmac_key_access_id": "example_string",
"hmac_key_secret": "example_string",
"keep_files_in_gcs_bucket": "example_string"
}
response = requests.post(url, headers=headers, json=data)
print(response.json())
const response = await fetch("https://mythic-analytics.gulp.workers.dev/client/v1/airbyte/config/provision", {
method: "POST",
headers: {
"Content-Type": "application/json",
"Authorization": "Bearer YOUR_API_TOKEN"
},
body: JSON.stringify({
"airbyte_workspace_id": "4fed5b02-4acd-4111-95bc-f7d15153b362",
"airbyte_destination_id": "613f34e2-bcc0-4bc5-ab3f-eeeff4647401",
"airbyte_client_id": "example_string",
"airbyte_client_secret": "example_string",
"confirm_shared_dataset": true,
"bigquery_project_id": "acme-analytics",
"bigquery_dataset": "mythic_ads",
"gcp_credentials_json": "example_string",
"dataset_location": "US",
"staging_bucket_name": "acme-airbyte-staging",
"staging_bucket_path": "example_string",
"hmac_key_access_id": "example_string",
"hmac_key_secret": "example_string",
"keep_files_in_gcs_bucket": "example_string"
})
});
const data = await response.json();
console.log(data);
package main
import (
"fmt"
"net/http"
"bytes"
"encoding/json"
)
func main() {
data := []byte(`{
"airbyte_workspace_id": "4fed5b02-4acd-4111-95bc-f7d15153b362",
"airbyte_destination_id": "613f34e2-bcc0-4bc5-ab3f-eeeff4647401",
"airbyte_client_id": "example_string",
"airbyte_client_secret": "example_string",
"confirm_shared_dataset": true,
"bigquery_project_id": "acme-analytics",
"bigquery_dataset": "mythic_ads",
"gcp_credentials_json": "example_string",
"dataset_location": "US",
"staging_bucket_name": "acme-airbyte-staging",
"staging_bucket_path": "example_string",
"hmac_key_access_id": "example_string",
"hmac_key_secret": "example_string",
"keep_files_in_gcs_bucket": "example_string"
}`)
req, err := http.NewRequest("POST", "https://mythic-analytics.gulp.workers.dev/client/v1/airbyte/config/provision", bytes.NewBuffer(data))
if err != nil {
panic(err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Bearer YOUR_API_TOKEN")
req.Header.Set("Authorization", "Bearer YOUR_API_TOKEN")
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
panic(err)
}
defer resp.Body.Close()
fmt.Println("Response Status:", resp.Status)
}
require 'net/http'
require 'json'
uri = URI('https://mythic-analytics.gulp.workers.dev/client/v1/airbyte/config/provision')
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
request = Net::HTTP::Post.new(uri)
request['Content-Type'] = 'application/json'
request['Authorization'] = 'Bearer YOUR_API_TOKEN'
request['Authorization'] = 'Bearer YOUR_API_TOKEN'
request.body = '{
"airbyte_workspace_id": "4fed5b02-4acd-4111-95bc-f7d15153b362",
"airbyte_destination_id": "613f34e2-bcc0-4bc5-ab3f-eeeff4647401",
"airbyte_client_id": "example_string",
"airbyte_client_secret": "example_string",
"confirm_shared_dataset": true,
"bigquery_project_id": "acme-analytics",
"bigquery_dataset": "mythic_ads",
"gcp_credentials_json": "example_string",
"dataset_location": "US",
"staging_bucket_name": "acme-airbyte-staging",
"staging_bucket_path": "example_string",
"hmac_key_access_id": "example_string",
"hmac_key_secret": "example_string",
"keep_files_in_gcs_bucket": "example_string"
}'
response = http.request(request)
puts response.body
{
"success": true,
"data": {
"workspace_id": "example_string",
"destination_id": "example_string",
"status": "active",
"loading_method": "Standard",
"destination_reused": true,
"adopted": {
"workspace": true,
"destination": true,
"airbyte_org": true
},
"notifications": {
"registered": true,
"changed": true,
"url": "example_string",
"events": [
"failure",
"success"
],
"replaced": "example_string",
"reason": "example_string"
}
}
}
{
"error": "Bad Request",
"message": "The request contains invalid parameters or malformed data",
"code": 400,
"details": [
{
"field": "email",
"message": "Invalid email format"
}
]
}
{
"error": "Unauthorized",
"message": "Authentication required. Please provide a valid API token",
"code": 401
}
{
"error": "Forbidden",
"message": "You don't have permission to access this resource",
"code": 403
}
{
"error": "Not Found",
"message": "The requested resource was not found",
"code": 404
}
{
"error": "Conflict",
"message": "The request conflicts with the current state of the resource",
"code": 409,
"details": "Resource already exists"
}
{
"error": "Error",
"message": "`provision_failed` — Airbyte rejected the request. The upstream reason is included in `error`.
",
"code": 502
}
{
"error": "Service Unavailable",
"message": "The service is temporarily unavailable. Please try again later",
"code": 503
}
/config/provision
Target server for requests. Edit to use your own host.
Agency key as bearer token, format Bearer ak_.... Grants full read-write access scoped to the agency. Agency-wide scoped keys (mcp_ with no fixed location) are accepted too and need airbyte:read or airbyte:write; a client-bound mcp_ key gets 403 agency_key_required. See Using an mcp_ key over HTTP.
Bearer ak_.... Grants full read-write access scoped to the agency. Agency-wide scoped keys (mcp_ with no fixed location) are accepted too and need airbyte:read or airbyte:write; a client-bound mcp_ key gets 403 agency_key_required. See Using an mcp_ key over HTTP.
Location secret key as bearer token, format Bearer sk_.... Grants read-only access; the agency is resolved from the location. Write endpoints return 403.
Bearer sk_.... Grants read-only access; the agency is resolved from the location. Write endpoints return 403.
The media type of the request body
ADOPT: an existing Airbyte workspace to create this agency's sources in, instead of creating a new one. Required alongside airbyte_destination_id — a destination can only be used by sources in its own workspace.
ADOPT: an existing BigQuery destination every connection writes through. This is what prevents two Airbyte workspaces syncing the same streams into the same dataset — duplicate rows, split incremental cursor state, and double Airbyte cost, none of which raises an error anywhere. Verified at provision time: a wrong id, or one in a different workspace, is rejected here rather than on the first connection. When supplied, bigquery_project_id, bigquery_dataset and gcp_credentials_json are not required.
ADOPT: your own Airbyte application client id. Required when the workspace/destination you are adopting lives in YOUR Airbyte org rather than Mythic's — Mythic's credentials cannot see it. Must be sent with airbyte_client_secret. Stored and used for every subsequent Airbyte call for this agency.
Paired with airbyte_client_id. Encrypted at rest; never returned.
Destination BigQuery project ID.
Destination BigQuery dataset.
GCP service-account credentials JSON, as a string. Encrypted at rest; never returned by the API.
BigQuery dataset location. Defaults to US.
GCS bucket for staged loads. ALL-OR-NOTHING: supply this together with hmac_key_access_id and hmac_key_secret to use GCS staging, or omit every staging_/hmac_ field to use standard inserts, which need no bucket and no HMAC key. A partial set is rejected with 400 staging_incomplete naming what is missing.
Path prefix within the staging bucket. Defaults to staging.
GCS HMAC key access ID. This is an interoperability credential created FOR a service account (gcloud storage hmac create SA_EMAIL) — not the service-account key itself. Required for GCS staging, together with the secret.
HMAC key secret. Required for GCS staging, together with the access ID.
Airbyte staging-file retention option. GCS staging only.
Request Preview
Response
Response will appear here after sending the request
Authentication
Bearer token. Agency key as bearer token, format Bearer ak_.... Grants full read-write access scoped to the agency. Agency-wide scoped keys (mcp_ with no fixed location) are accepted too and need airbyte:read or airbyte:write; a client-bound mcp_ key gets 403 agency_key_required. See Using an mcp_ key over HTTP.
Bearer token. Location secret key as bearer token, format Bearer sk_.... Grants read-only access; the agency is resolved from the location. Write endpoints return 403.
Body
ADOPT: an existing Airbyte workspace to create this agency's sources in, instead of creating a new one. Required alongside airbyte_destination_id — a destination can only be used by sources in its own workspace.
4fed5b02-4acd-4111-95bc-f7d15153b362ADOPT: an existing BigQuery destination every connection writes through. This is what prevents two Airbyte workspaces syncing the same streams into the same dataset — duplicate rows, split incremental cursor state, and double Airbyte cost, none of which raises an error anywhere. Verified at provision time: a wrong id, or one in a different workspace, is rejected here rather than on the first connection. When supplied, bigquery_project_id, bigquery_dataset and gcp_credentials_json are not required.
613f34e2-bcc0-4bc5-ab3f-eeeff4647401ADOPT: your own Airbyte application client id. Required when the workspace/destination you are adopting lives in YOUR Airbyte org rather than Mythic's — Mythic's credentials cannot see it. Must be sent with airbyte_client_secret. Stored and used for every subsequent Airbyte call for this agency.
Paired with airbyte_client_id. Encrypted at rest; never returned.
GCP service-account credentials JSON, as a string. Encrypted at rest; never returned by the API.
GCS bucket for staged loads. ALL-OR-NOTHING: supply this together with hmac_key_access_id and hmac_key_secret to use GCS staging, or omit every staging_/hmac_ field to use standard inserts, which need no bucket and no HMAC key. A partial set is rejected with 400 staging_incomplete naming what is missing.
acme-airbyte-stagingPath prefix within the staging bucket. Defaults to staging.
GCS HMAC key access ID. This is an interoperability credential created FOR a service account (gcloud storage hmac create SA_EMAIL) — not the service-account key itself. Required for GCS staging, together with the secret.
HMAC key secret. Required for GCS staging, together with the access ID.
Airbyte staging-file retention option. GCS staging only.