{ "openapi": "3.0.3", "info": { "title": "Crawlwise Data API", "version": "1.0.0", "description": "Website change alerts via signed webhooks, plus on-demand tech-stack, feed and sitemap APIs.\nDeterministic Node.js — no browser, no AI at runtime. Pay for changes, not empty checks.\n\nHuman docs: https://crawlwisedata.com/docs.html\nMCP: local stdio package `crawlwise-mcp` on npm; install with `npx crawlwise-mcp`.\nAgent index: https://crawlwisedata.com/llms.txt\n\nAuth: `Authorization: Bearer cw_live_...` or `X-API-Key`. Demo (no key): 20 calls/day/IP on data endpoints (Retry-After until 00:00 UTC).\nFree trial key: `POST /v1/trial` with `{\"email\":\"you@company.com\",\"acceptTerms\":true}` returns a key with 1,000 one-time credits (one per email every 30 days; per network 5 requests/hour and 10/day; disposable domains refused; if a call costs more than the balance it is refused and nothing is charged).\nFree account (0 credits): `POST /v1/signup` with `{\"email\":\"…\"}` emails a verification code; `POST /v1/signup/verify` with the code issues the API key; then `POST /v1/terms/accept` with `{\"acceptTerms\":true,\"termsVersion\":\"…\"}` activates it (until then the key can only read the account). Purchases are non-refundable except where law requires.\nWebhooks: HMAC-SHA256 header `X-Crawlwise-Signature: t=,v1=`.\nSee a change during a trial: `POST /v1/monitors/test` sends a free signed sample event to your webhookUrl; monitor the built-in demo target (`/demo-target/feed.xml` or `/demo-target/sitemap.xml`, changes every 5 minutes) and call `POST /v1/monitors/run` to check now. A sitemap monitor baseline costs a flat 10 credits for any site size.", "contact": { "name": "Crawlwise Data support", "email": "crawlwise.support@gmail.com", "url": "https://crawlwisedata.com" }, "license": { "name": "GPL-3.0", "url": "https://www.gnu.org/licenses/gpl-3.0.html" } }, "servers": [ { "url": "https://crawlwisedata.com", "description": "Production" } ], "tags": [ { "name": "Data", "description": "On-demand lookups (techstack, feed, sitemap)" }, { "name": "Monitors", "description": "Named monitors and server-side schedules" }, { "name": "Webhooks", "description": "Signing secret and delivery log" }, { "name": "Account", "description": "Balance, keys, plan" }, { "name": "Trial", "description": "Free trial key (no card, one per email every 30 days; per network 5 requests/hour and 10/day)" }, { "name": "Signup", "description": "Free pre-pay account with email verification and ToS acceptance" }, { "name": "Public", "description": "Pricing, health, status (no auth, CORS enabled)" }, { "name": "Webhook events", "description": "Outbound POSTs Crawlwise sends to your URL" } ], "paths": { "/v1/techstack": { "get": { "tags": [ "Data" ], "operationId": "getTechstack", "summary": "Detect website tech stack", "description": "30 credits per successfully analyzed page. Demo works without a key. SaaS domain-verification DNS TXT records (openai-domain-verification=…, MS=…, stripe-verification=…) are NOT technologies: they are returned in data.verifications with confidence 0 and never appear in technologies, technologyNames or categories. SPF includes (v=spf1 include:sendgrid.net) still count as mail-sending technologies.", "security": [ { "BearerAuth": [] }, { "ApiKeyAuth": [] }, {} ], "parameters": [ { "name": "url", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Page or site URL." }, { "name": "dns", "in": "query", "schema": { "type": "boolean", "default": true }, "description": "Include DNS/MX/NS/TXT signals." }, { "name": "minConfidence", "in": "query", "schema": { "type": "integer", "minimum": 0, "maximum": 100, "default": 0 } }, { "name": "implied", "in": "query", "schema": { "type": "boolean", "default": true }, "description": "Include technologies implied by others." } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/DataEnvelope" } } } }, "400": { "description": "Bad request (invalid_params, invalid_url, unsupported_url)", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "401": { "description": "Missing or invalid key", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "402": { "description": "Insufficient credits", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "403": { "description": "Account suspended or robots.txt disallows", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "404": { "description": "Not found", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "422": { "description": "Target reachable but no data / target error", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "429": { "description": "Rate limited", "headers": { "Retry-After": { "schema": { "type": "integer" } } }, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "500": { "description": "Internal error (not charged)", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "504": { "description": "Timeout (not charged)", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } } } } }, "/v1/feed": { "get": { "tags": [ "Data" ], "operationId": "getFeed", "summary": "Read RSS/Atom/JSON Feed (optionally new-items-only)", "description": "10 credits per successful read. With monitor=NAME returns only unseen items after the baseline.", "security": [ { "BearerAuth": [] }, { "ApiKeyAuth": [] }, {} ], "parameters": [ { "name": "url", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Feed URL or website URL (autodiscovery)." }, { "name": "max", "in": "query", "schema": { "type": "integer", "minimum": 1, "maximum": 200, "default": 50 } }, { "name": "include", "in": "query", "schema": { "type": "array", "items": { "type": "string" } }, "style": "form", "explode": true }, { "name": "exclude", "in": "query", "schema": { "type": "array", "items": { "type": "string" } }, "style": "form", "explode": true }, { "name": "publishedAfter", "in": "query", "schema": { "type": "string" }, "description": "Date (2026-09-01) or relative (7 days)." }, { "name": "content", "in": "query", "schema": { "type": "boolean", "default": false } }, { "name": "monitor", "in": "query", "schema": { "type": "string", "pattern": "^[A-Za-z0-9_-]{1,40}$" }, "description": "Keys only. New-items mode." } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/DataEnvelope" } } } }, "400": { "description": "Bad request (invalid_params, invalid_url, unsupported_url)", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "401": { "description": "Missing or invalid key", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "402": { "description": "Insufficient credits", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "403": { "description": "Account suspended or robots.txt disallows", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "404": { "description": "Not found", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "422": { "description": "Target reachable but no data / target error", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "429": { "description": "Rate limited", "headers": { "Retry-After": { "schema": { "type": "integer" } } }, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "500": { "description": "Internal error (not charged)", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "504": { "description": "Timeout (not charged)", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } } } } }, "/v1/sitemap": { "get": { "tags": [ "Data" ], "operationId": "getSitemap", "summary": "Extract sitemap URLs (optionally change-diff)", "description": "10 credits + 1 per URL returned. With monitor=NAME returns new/updated/removed since last call. The first call for a monitor (baseline) costs a flat 10 credits for any site size (only the credits for the flat fee are reserved, so a trial can baseline a 10,000-URL sitemap) and returns counts + a 20-URL sample (monitor.urlsInSnapshot, monitor.baselinePricing=\"flat\"); 50 flat baselines per account per day (10 on a trial), then per-URL pricing. Truncated results never save a snapshot and never report \"removed\".", "security": [ { "BearerAuth": [] }, { "ApiKeyAuth": [] }, {} ], "parameters": [ { "name": "url", "in": "query", "required": true, "schema": { "type": "string" } }, { "name": "max", "in": "query", "schema": { "type": "integer", "minimum": 1, "maximum": 10000, "default": 1000 } }, { "name": "include", "in": "query", "schema": { "type": "array", "items": { "type": "string" } }, "style": "form", "explode": true, "description": "Case-insensitive regex filters." }, { "name": "exclude", "in": "query", "schema": { "type": "array", "items": { "type": "string" } }, "style": "form", "explode": true }, { "name": "lastmodAfter", "in": "query", "schema": { "type": "string" } }, { "name": "monitor", "in": "query", "schema": { "type": "string", "pattern": "^[A-Za-z0-9_-]{1,40}$" } } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/DataEnvelope" } } } }, "400": { "description": "Bad request (invalid_params, invalid_url, unsupported_url)", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "401": { "description": "Missing or invalid key", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "402": { "description": "Insufficient credits", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "403": { "description": "Account suspended or robots.txt disallows", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "404": { "description": "Not found", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "422": { "description": "Target reachable but no data / target error", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "429": { "description": "Rate limited", "headers": { "Retry-After": { "schema": { "type": "integer" } } }, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "500": { "description": "Internal error (not charged)", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "504": { "description": "Timeout (not charged)", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } } } } }, "/v1/monitors": { "get": { "tags": [ "Monitors" ], "operationId": "listMonitors", "summary": "List monitors", "security": [ { "BearerAuth": [] }, { "ApiKeyAuth": [] } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "type": "object", "properties": { "ok": { "type": "boolean" }, "count": { "type": "integer" }, "monitors": { "type": "array", "items": { "$ref": "#/components/schemas/Monitor" } }, "scheduling": { "type": "object", "additionalProperties": true } } } } } }, "401": { "description": "Missing or invalid key", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } } } }, "post": { "tags": [ "Monitors" ], "operationId": "createMonitor", "summary": "Create or update a monitor", "description": "With schedule:true and https webhookUrl, Crawlwise runs checks and POSTs signed change events.", "security": [ { "BearerAuth": [] }, { "ApiKeyAuth": [] } ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "type": "object", "required": [ "endpoint", "name", "url" ], "properties": { "endpoint": { "type": "string", "enum": [ "feed", "sitemap", "techstack" ] }, "name": { "type": "string", "pattern": "^[A-Za-z0-9_-]{1,40}$" }, "url": { "type": "string" }, "webhookUrl": { "type": "string", "nullable": true }, "schedule": { "type": "boolean" }, "enabled": { "type": "boolean" }, "intervalSec": { "type": "integer", "minimum": 900, "maximum": 86400, "description": "Desired seconds between scheduled checks. Minimum 900 (15 minutes) on every plan; lower values are raised to 900. Scheduled checks run usually within an hour, plus check-now on demand." }, "max": { "type": "integer" }, "include": { "type": "array", "items": { "type": "string" } }, "exclude": { "type": "array", "items": { "type": "string" } }, "content": { "type": "boolean" }, "publishedAfter": { "type": "string" }, "lastmodAfter": { "type": "string" }, "dns": { "type": "boolean" }, "minConfidence": { "type": "integer" }, "implied": { "type": "boolean" } } } } } }, "responses": { "200": { "description": "Updated", "content": { "application/json": { "schema": { "type": "object", "properties": { "ok": { "type": "boolean" }, "created": { "type": "boolean" }, "monitor": { "$ref": "#/components/schemas/Monitor" }, "note": { "type": "string" } } } } } }, "201": { "description": "Created", "content": { "application/json": { "schema": { "type": "object", "properties": { "ok": { "type": "boolean" }, "created": { "type": "boolean" }, "monitor": { "$ref": "#/components/schemas/Monitor" }, "note": { "type": "string" } } } } } }, "400": { "description": "Bad request (invalid_params, invalid_url, unsupported_url)", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "401": { "description": "Missing or invalid key", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } } } }, "delete": { "tags": [ "Monitors" ], "operationId": "deleteMonitor", "summary": "Delete a monitor", "security": [ { "BearerAuth": [] }, { "ApiKeyAuth": [] } ], "parameters": [ { "name": "endpoint", "in": "query", "required": true, "schema": { "type": "string", "enum": [ "feed", "sitemap", "techstack" ] } }, { "name": "name", "in": "query", "required": true, "schema": { "type": "string" } } ], "responses": { "200": { "description": "Deleted", "content": { "application/json": { "schema": { "type": "object", "properties": { "ok": { "type": "boolean" }, "deleted": { "type": "object" } } } } } }, "401": { "description": "Missing or invalid key", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "404": { "description": "Not found", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } } } } }, "/v1/monitors/test": { "post": { "tags": [ "Monitors" ], "operationId": "sendMonitorTestEvent", "summary": "Send a signed sample webhook to the monitor webhookUrl (free)", "description": "Delivers a `monitor.test` event (synthetic change shaped like the monitor endpoint) to the saved webhookUrl, signed with your account secret exactly like real events, plus header `X-Crawlwise-Test: 1`. One attempt, 8 s timeout, redirects not followed, non-public addresses refused. 0 credits. 10 per hour per account. The response echoes the exact body and signature header that were sent so you can check your verifier.", "security": [ { "BearerAuth": [] }, { "ApiKeyAuth": [] } ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/MonitorRunRequest" } } } }, "responses": { "200": { "description": "Attempted (see delivered/status)", "content": { "application/json": { "schema": { "type": "object", "additionalProperties": true, "properties": { "ok": { "type": "boolean" }, "test": { "type": "boolean" }, "delivered": { "type": "boolean" }, "status": { "type": "integer", "nullable": true }, "error": { "type": "string", "nullable": true }, "deliveryId": { "type": "string" }, "eventId": { "type": "string" }, "charged": { "$ref": "#/components/schemas/Charged" }, "sent": { "type": "object", "properties": { "headers": { "type": "object", "additionalProperties": true }, "body": { "type": "string" } } }, "verify": { "type": "string" } } } } } }, "400": { "description": "Monitor has no webhookUrl / invalid body", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "401": { "description": "Missing or invalid key", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "404": { "description": "Not found", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "429": { "description": "Rate limited", "headers": { "Retry-After": { "schema": { "type": "integer" } } }, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } } } } }, "/v1/monitors/run": { "post": { "tags": [ "Monitors" ], "operationId": "runMonitorNow", "summary": "Check a monitor now (and deliver a signed webhook if it changed)", "description": "Runs one check immediately with the same billing as a scheduled check (baseline: feed 10, sitemap flat 10, techstack 30; no change: 2 / 5; change: full price, sitemap 10 + 1 per changed URL; delivering a sitemap change needs at least 11 credits (10 base + 1 per URL); a check that finds no change costs 2; carried-over URLs 1 each, no second base fee). If something changed and the monitor has a webhookUrl, a real signed `monitor.change` is POSTed (one attempt). Bound by the minimum monitor interval, the same on every plan (429 + Retry-After), and 30 runs/hour per account. Works for monitors with or without schedule:true. Failed checks are not charged.", "security": [ { "BearerAuth": [] }, { "ApiKeyAuth": [] } ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/MonitorRunRequest" } } } }, "responses": { "200": { "description": "Checked", "content": { "application/json": { "schema": { "type": "object", "additionalProperties": true, "properties": { "ok": { "type": "boolean" }, "run": { "type": "object", "additionalProperties": true, "properties": { "baseline": { "type": "boolean" }, "changed": { "type": "boolean" }, "credits": { "type": "integer" }, "status": { "type": "string" }, "change": { "type": "object", "nullable": true, "additionalProperties": true }, "delivery": { "type": "object", "nullable": true, "additionalProperties": true } } }, "charged": { "$ref": "#/components/schemas/Charged" }, "balance": { "$ref": "#/components/schemas/Balance" }, "note": { "type": "string" } } } } } }, "400": { "description": "Bad request (invalid_params, invalid_url, unsupported_url)", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "401": { "description": "Missing or invalid key", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "402": { "description": "Insufficient credits", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "404": { "description": "Not found", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "422": { "description": "Target reachable but no data / target error", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "429": { "description": "Rate limited", "headers": { "Retry-After": { "schema": { "type": "integer" } } }, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } } } } }, "/v1/demo-target": { "get": { "tags": [ "Public" ], "operationId": "getDemoTarget", "summary": "Built-in demo target that changes every 5 minutes", "description": "URLs of a feed (`/demo-target/feed.xml`: one new item every 5 minutes) and a sitemap (`/demo-target/sitemap.xml`: one new URL, one removed, one updated every 5 minutes) on this host. Point a monitor at them to see a real diff and webhook during a trial. Monitor checks on them are billed normally.", "security": [], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "type": "object", "properties": { "ok": { "type": "boolean" }, "feed": { "type": "string" }, "sitemap": { "type": "string" }, "changesEverySec": { "type": "integer", "example": 300 }, "what": { "type": "string" } } } } } } } } }, "/v1/webhooks/secret": { "get": { "tags": [ "Webhooks" ], "operationId": "getWebhookSecret", "summary": "Get (or create) webhook signing secret", "security": [ { "BearerAuth": [] }, { "ApiKeyAuth": [] } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "type": "object", "properties": { "ok": { "type": "boolean" }, "secret": { "type": "string", "description": "64 hex chars" }, "header": { "type": "string", "example": "X-Crawlwise-Signature" }, "format": { "type": "string" }, "note": { "type": "string" } } } } } }, "401": { "description": "Missing or invalid key", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } } } } }, "/v1/webhooks/secret/rotate": { "post": { "tags": [ "Webhooks" ], "operationId": "rotateWebhookSecret", "summary": "Rotate webhook signing secret", "security": [ { "BearerAuth": [] }, { "ApiKeyAuth": [] } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "type": "object", "properties": { "ok": { "type": "boolean" }, "secret": { "type": "string" }, "note": { "type": "string" } } } } } }, "401": { "description": "Missing or invalid key", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } } } } }, "/v1/webhooks/deliveries": { "get": { "tags": [ "Webhooks" ], "operationId": "listWebhookDeliveries", "summary": "Recent webhook delivery attempts", "security": [ { "BearerAuth": [] }, { "ApiKeyAuth": [] } ], "parameters": [ { "name": "limit", "in": "query", "schema": { "type": "integer", "minimum": 1, "maximum": 100, "default": 25 } } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "type": "object", "properties": { "ok": { "type": "boolean" }, "count": { "type": "integer" }, "deliveries": { "type": "array", "items": { "$ref": "#/components/schemas/Delivery" } } } } } } }, "401": { "description": "Missing or invalid key", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } } } } }, "/v1/account": { "get": { "tags": [ "Account" ], "operationId": "getAccount", "summary": "Account balance, plan, keys, top-up links", "security": [ { "BearerAuth": [] }, { "ApiKeyAuth": [] } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "type": "object", "additionalProperties": true, "properties": { "ok": { "type": "boolean" }, "accountId": { "type": "string" }, "accountType": { "type": "string", "enum": [ "trial", "standard" ], "description": "\"trial\" for free trial keys until credits are purchased" }, "trial": { "type": "object", "nullable": true, "additionalProperties": true, "properties": { "active": { "type": "boolean" }, "label": { "type": "string", "example": "FREE TRIAL KEY" }, "issuedAt": { "type": "string", "format": "date-time" }, "creditsGranted": { "type": "integer" }, "creditsLeft": { "type": "integer" }, "canGoNegative": { "type": "boolean", "enum": [ false ] } } }, "status": { "type": "string" }, "balance": { "type": "object", "additionalProperties": true }, "plan": { "type": "object", "additionalProperties": true }, "keys": { "type": "array", "items": { "type": "object" } } } } } } }, "401": { "description": "Missing or invalid key", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } } } } }, "/v1/trial": { "get": { "tags": [ "Trial" ], "operationId": "getTrialInfo", "summary": "Free trial rules (enabled, credits, limits)", "security": [], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "type": "object", "additionalProperties": true, "properties": { "ok": { "type": "boolean" }, "enabled": { "type": "boolean" }, "credits": { "type": "integer", "example": 1000 }, "endpoint": { "type": "string" }, "rules": { "type": "array", "items": { "type": "string" } } } } } } } } }, "post": { "tags": [ "Trial" ], "operationId": "createTrialKey", "summary": "Get a free trial API key (shown once)", "description": "Issues one API key with 1,000 one-time credits. No auth. One per normalized email (case, +tags and Gmail dots ignored) every 30 days; soft caps per IP and per network for shared egress; 5 attempts per hour per IP; global daily cap. Disposable/reserved email domains and domains without MX records are refused. Trial keys have lower limits (20 req/min, 2 concurrent, 5 monitors, 1 active key) and can never go below zero (402 when a call needs more credits than are left). Buying a pack or plan on the account ends the trial. Terms: send `acceptTerms: true` to accept the Terms of Service and Privacy Policy at issuance; otherwise the key is issued with `termsRequired: true` and every call answers 403 `terms_required` until you call `POST /v1/terms/accept` with the key. We never send email.", "security": [], "requestBody": { "required": true, "content": { "application/json": { "schema": { "type": "object", "required": [ "email" ], "properties": { "email": { "type": "string", "format": "email", "maxLength": 254, "example": "you@company.com" }, "acceptTerms": { "type": "boolean", "example": true, "description": "true = you accept the Terms of Service and Privacy Policy now. If omitted, accept later with POST /v1/terms/accept before using the key." }, "termsVersion": { "type": "string", "description": "Optional; must equal the current Terms version if sent." } } } } } }, "responses": { "201": { "description": "Trial key issued", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/TrialKey" } } } }, "400": { "description": "Invalid email, disposable domain (reason disposable_email) or domain without mail (reason domain_has_no_mail)", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "409": { "description": "trial_not_available: a trial was already issued for this email, IP or network in the last 30 days (error.reason = email | ip | network)", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "429": { "description": "Rate limited", "headers": { "Retry-After": { "schema": { "type": "integer" } } }, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "503": { "description": "Trials disabled", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } } } } }, "/v1/signup": { "get": { "tags": [ "Signup" ], "operationId": "getSignupInfo", "summary": "Free account signup rules (ToS version, mail status, OAuth availability)", "security": [], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "type": "object", "additionalProperties": true } } } } } }, "post": { "tags": [ "Signup" ], "operationId": "startSignup", "summary": "Start free account signup (emails verification code; no API key yet)", "description": "Sends a 6-digit code to the email. The API key is issued only after POST /v1/signup/verify. Rate-limited. Disposable emails refused.", "security": [], "requestBody": { "required": true, "content": { "application/json": { "schema": { "type": "object", "required": [ "email" ], "properties": { "email": { "type": "string", "format": "email" } } } } } }, "responses": { "201": { "description": "Code emailed", "content": { "application/json": { "schema": { "type": "object", "additionalProperties": true, "properties": { "ok": { "type": "boolean" }, "pending": { "type": "boolean" }, "pendingId": { "type": "string" }, "email": { "type": "string" }, "expiresAt": { "type": "string", "format": "date-time" } } } } } }, "400": { "description": "Invalid email", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "429": { "description": "Rate limited", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "503": { "description": "Mail not configured", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } } } } }, "/v1/terms/accept": { "post": { "operationId": "acceptTerms", "summary": "Accept the Terms of Service (required after email verification before the key can be used)", "description": "Explicit clickwrap tied to the account. Records the time, Terms version, and how it was accepted. Until accepted, accounts created by email sign-up get 403 terms_required on paid endpoints.", "requestBody": { "required": true, "content": { "application/json": { "schema": { "type": "object", "required": [ "acceptTerms" ], "properties": { "acceptTerms": { "type": "boolean", "enum": [ true ] }, "termsVersion": { "type": "string" } } } } } }, "responses": { "200": { "description": "Accepted", "content": { "application/json": { "schema": { "type": "object", "additionalProperties": true, "properties": { "ok": { "type": "boolean" }, "acceptedTermsAt": { "type": "string", "format": "date-time" }, "termsVersion": { "type": "string" } } } } } }, "400": { "description": "acceptTerms missing or stale termsVersion", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "401": { "description": "Missing or invalid key/session", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } } } } }, "/v1/signup/verify": { "post": { "tags": [ "Signup" ], "operationId": "verifySignup", "summary": "Verify email code and issue API key (shown once)", "security": [], "requestBody": { "required": true, "content": { "application/json": { "schema": { "type": "object", "required": [ "pendingId", "code" ], "properties": { "pendingId": { "type": "string" }, "code": { "type": "string", "pattern": "^[0-9]{6}$" } } } } } }, "responses": { "201": { "description": "Account created", "content": { "application/json": { "schema": { "type": "object", "additionalProperties": true, "properties": { "ok": { "type": "boolean" }, "apiKey": { "type": "string" }, "accountId": { "type": "string" }, "credits": { "type": "integer", "enum": [ 0 ] } } } } } }, "400": { "description": "Bad or expired code", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, "409": { "description": "signup_not_available", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } } } } }, "/v1/signup/resend": { "post": { "tags": [ "Signup" ], "operationId": "resendSignupCode", "summary": "Resend verification code (rate-limited)", "security": [], "requestBody": { "required": true, "content": { "application/json": { "schema": { "type": "object", "required": [ "pendingId" ], "properties": { "pendingId": { "type": "string" } } } } } }, "responses": { "200": { "description": "Code resent", "content": { "application/json": { "schema": { "type": "object", "additionalProperties": true } } } }, "429": { "description": "Rate limited", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } } } } }, "/v1/oauth/providers": { "get": { "tags": [ "Signup" ], "operationId": "oauthProviders", "summary": "Which social login providers are configured", "security": [], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "type": "object", "additionalProperties": true } } } } } } }, "/v1/pricing": { "get": { "tags": [ "Public" ], "operationId": "getPricing", "summary": "Machine-readable prices, packs, plans, limits", "security": [], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "type": "object", "additionalProperties": true } } } } } } }, "/v1/health": { "get": { "tags": [ "Public" ], "operationId": "getHealth", "summary": "Liveness + storage probe", "security": [], "responses": { "200": { "description": "Healthy", "content": { "application/json": { "schema": { "type": "object", "properties": { "ok": { "type": "boolean" }, "storage": { "type": "string" }, "storageMs": { "type": "number" }, "time": { "type": "string" } } } } } }, "503": { "description": "Storage unhealthy", "content": { "application/json": { "schema": { "type": "object", "additionalProperties": true } } } } } } }, "/v1/status": { "get": { "tags": [ "Public" ], "operationId": "getStatus", "summary": "Component status", "security": [], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "type": "object", "additionalProperties": true } } } } } } } }, "components": { "securitySchemes": { "BearerAuth": { "type": "http", "scheme": "bearer", "bearerFormat": "API key", "description": "API key as `Authorization: Bearer cw_live_...` (or `X-API-Key`)." }, "ApiKeyAuth": { "type": "apiKey", "in": "header", "name": "X-API-Key", "description": "Alternative to Bearer auth." } }, "schemas": { "Error": { "type": "object", "properties": { "ok": { "type": "boolean", "enum": [ false ] }, "error": { "type": "object", "properties": { "code": { "type": "string", "example": "invalid_params" }, "message": { "type": "string" }, "billable": { "type": "boolean", "example": false } }, "required": [ "code", "message" ] } } }, "DataEnvelope": { "type": "object", "required": [ "ok", "endpoint", "mode", "charged", "data" ], "properties": { "ok": { "type": "boolean", "enum": [ true ] }, "endpoint": { "type": "string", "enum": [ "techstack", "feed", "sitemap" ] }, "mode": { "type": "string", "enum": [ "paid", "demo", "marketplace" ] }, "charged": { "type": "object", "properties": { "credits": { "type": "integer" }, "usd": { "type": "number" } } }, "balance": { "type": "object", "nullable": true, "properties": { "credits": { "type": "integer" }, "usd": { "type": "number" } } }, "data": { "type": "object", "additionalProperties": true } } }, "Charged": { "type": "object", "properties": { "credits": { "type": "integer" }, "usd": { "type": "number" } } }, "Balance": { "type": "object", "nullable": true, "properties": { "credits": { "type": "integer" }, "usd": { "type": "number" } } }, "Verification": { "type": "object", "description": "A SaaS domain-verification DNS TXT record (e.g. openai-domain-verification=…). Shows an account was set up for the domain, NOT that the site uses the product. Never listed in technologies/technologyNames.", "properties": { "name": { "type": "string", "example": "OpenAI" }, "confidence": { "type": "integer", "enum": [ 0 ] }, "signal": { "type": "string", "enum": [ "dns-txt-domain-verification" ] }, "record": { "type": "string", "enum": [ "TXT" ] }, "categories": { "type": "array", "items": { "type": "string" } }, "website": { "type": "string", "nullable": true }, "alsoDetectedAsTechnology": { "type": "boolean", "description": "true if real signals (page, headers, MX/SPF, …) also detected it" } } }, "MonitorRunRequest": { "type": "object", "required": [ "endpoint", "name" ], "properties": { "endpoint": { "type": "string", "enum": [ "feed", "sitemap", "techstack" ] }, "name": { "type": "string", "pattern": "^[A-Za-z0-9_-]{1,40}$" } } }, "MonitorTestEvent": { "type": "object", "description": "Sample event POSTed by POST /v1/monitors/test. Same envelope, headers and signature as monitor.change, plus `X-Crawlwise-Test: 1`. The change is synthetic.", "required": [ "id", "type", "test", "created_at", "account_id", "monitor", "change" ], "properties": { "id": { "type": "string", "example": "evt_test_..." }, "type": { "type": "string", "enum": [ "monitor.test" ] }, "test": { "type": "boolean", "enum": [ true ] }, "created_at": { "type": "string", "format": "date-time" }, "detected_at": { "type": "string", "format": "date-time" }, "account_id": { "type": "string" }, "monitor": { "type": "object", "properties": { "endpoint": { "type": "string", "enum": [ "feed", "sitemap", "techstack" ] }, "name": { "type": "string" }, "url": { "type": "string", "nullable": true } } }, "change": { "oneOf": [ { "$ref": "#/components/schemas/WebhookChangeFeed" }, { "$ref": "#/components/schemas/WebhookChangeSitemap" }, { "$ref": "#/components/schemas/WebhookChangeTech" } ] }, "credits": { "type": "object", "properties": { "charged": { "type": "integer", "enum": [ 0 ] } } }, "note": { "type": "string" } } }, "TrialKey": { "type": "object", "required": [ "ok", "trial", "accountId", "apiKey", "credits" ], "properties": { "ok": { "type": "boolean", "enum": [ true ] }, "trial": { "type": "boolean", "enum": [ true ] }, "accountId": { "type": "string", "example": "acct_..." }, "keyId": { "type": "string", "example": "key_..." }, "apiKey": { "type": "string", "description": "Shown once. Only a hash is stored.", "example": "cw_test_..." }, "credits": { "type": "integer", "example": 1000 }, "note": { "type": "string" }, "limits": { "type": "object", "additionalProperties": true }, "upgrade": { "type": "string" } } }, "Monitor": { "type": "object", "properties": { "endpoint": { "type": "string", "enum": [ "feed", "sitemap", "techstack" ] }, "name": { "type": "string", "pattern": "^[A-Za-z0-9_-]{1,40}$" }, "target": { "type": "string", "format": "uri" }, "url": { "type": "string", "format": "uri" }, "webhookUrl": { "type": "string", "format": "uri", "nullable": true }, "schedule": { "type": "boolean" }, "enabled": { "type": "boolean" }, "intervalSec": { "type": "integer", "minimum": 900, "description": "Seconds between scheduled checks (at least 900 on every plan; older monitors with a shorter interval are treated as 900)." }, "options": { "type": "object", "additionalProperties": true }, "createdAt": { "type": "string", "format": "date-time" }, "updatedAt": { "type": "string", "format": "date-time" }, "lastRunAt": { "type": "string", "format": "date-time", "nullable": true }, "nextRunAt": { "type": "string", "format": "date-time", "nullable": true }, "lastStatus": { "type": "string", "nullable": true }, "lastChangeAt": { "type": "string", "format": "date-time", "nullable": true }, "lastError": { "type": "string", "nullable": true }, "lastEventId": { "type": "string", "nullable": true }, "lastDeliveryId": { "type": "string", "nullable": true }, "lastCharged": { "type": "integer", "nullable": true } } }, "Delivery": { "type": "object", "properties": { "id": { "type": "string" }, "eventId": { "type": "string" }, "type": { "type": "string", "enum": [ "monitor.change", "monitor.test" ], "nullable": true }, "accountId": { "type": "string" }, "webhookUrl": { "type": "string" }, "attempt": { "type": "integer" }, "status": { "type": "integer" }, "ok": { "type": "boolean" }, "ms": { "type": "integer" }, "at": { "type": "string", "format": "date-time" }, "monitor": { "type": "object", "additionalProperties": true }, "error": { "type": "string", "nullable": true } } }, "MonitorChangeEvent": { "type": "object", "description": "Signed JSON body POSTed to your webhookUrl when a scheduled monitor detects a change. Verify X-Crawlwise-Signature before acting.", "required": [ "id", "type", "created_at", "account_id", "monitor", "change" ], "properties": { "id": { "type": "string", "description": "Event id (also in X-Crawlwise-Event header)" }, "type": { "type": "string", "enum": [ "monitor.change" ] }, "created_at": { "type": "string", "format": "date-time" }, "detected_at": { "type": "string", "format": "date-time" }, "account_id": { "type": "string" }, "monitor": { "type": "object", "properties": { "endpoint": { "type": "string", "enum": [ "feed", "sitemap", "techstack" ] }, "name": { "type": "string" }, "url": { "type": "string", "nullable": true } } }, "change": { "oneOf": [ { "type": "object", "properties": { "kind": { "type": "string", "enum": [ "feed.new_items" ] }, "newItems": { "type": "integer" }, "items": { "type": "array", "items": { "type": "object", "additionalProperties": true } } } }, { "type": "object", "properties": { "kind": { "type": "string", "enum": [ "sitemap.diff" ] }, "counts": { "type": "object", "properties": { "new": { "type": "integer" }, "updated": { "type": "integer" }, "removed": { "type": "integer" }, "unchanged": { "type": "integer" } } }, "changes": { "type": "array", "items": { "type": "object", "additionalProperties": true } } } }, { "type": "object", "properties": { "kind": { "type": "string", "enum": [ "techstack.diff" ] }, "added": { "type": "array", "items": { "type": "string" } }, "removed": { "type": "array", "items": { "type": "string" } }, "technologyNames": { "type": "array", "items": { "type": "string" } } } } ], "discriminator": { "propertyName": "kind" } }, "credits": { "type": "object", "properties": { "charged": { "type": "integer" } } } } }, "WebhookChangeFeed": { "type": "object", "properties": { "kind": { "type": "string", "enum": [ "feed.new_items" ] }, "newItems": { "type": "integer" }, "items": { "type": "array", "items": { "type": "object", "additionalProperties": true } } } }, "WebhookChangeSitemap": { "type": "object", "properties": { "kind": { "type": "string", "enum": [ "sitemap.diff" ] }, "counts": { "type": "object", "properties": { "new": { "type": "integer" }, "updated": { "type": "integer" }, "removed": { "type": "integer" }, "unchanged": { "type": "integer" } } }, "changes": { "type": "array", "items": { "type": "object", "additionalProperties": true } } } }, "WebhookChangeTech": { "type": "object", "properties": { "kind": { "type": "string", "enum": [ "techstack.diff" ] }, "added": { "type": "array", "items": { "type": "string" } }, "removed": { "type": "array", "items": { "type": "string" } }, "technologyNames": { "type": "array", "items": { "type": "string" } } } } } }, "x-webhooks": { "monitorChange": { "post": { "tags": [ "Webhook events" ], "operationId": "webhookMonitorChange", "summary": "Crawlwise → your server: monitor.change", "description": "POSTed to the monitor webhookUrl when a scheduled check finds a change.\nHeaders: `X-Crawlwise-Signature`, `X-Crawlwise-Delivery`, `X-Crawlwise-Timestamp`, `X-Crawlwise-Event`, `User-Agent: Crawlwise-Webhooks/1.0`.\nVerify signature: HMAC-SHA256 of `` `${t}.${rawBody}` `` with your secret from GET /v1/webhooks/secret; compare to v1 hex (timing-safe). Reject if |now - t| > 300s.\nReturn 2xx within 10s. Crawlwise retries up to 3 times with backoff (scheduled checks; POST /v1/monitors/run makes one attempt). Redirects are not followed.", "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/MonitorChangeEvent" } } } }, "responses": { "200": { "description": "Acknowledge delivery" } } } }, "monitorTest": { "post": { "tags": [ "Webhook events" ], "operationId": "webhookMonitorTest", "summary": "Crawlwise → your server: monitor.test (from POST /v1/monitors/test)", "description": "Same headers and signature as monitor.change, plus `X-Crawlwise-Test: 1`. Body type is \"monitor.test\" with test=true and a synthetic change. Do not act on it as a real change.", "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/MonitorTestEvent" } } } }, "responses": { "200": { "description": "Acknowledge delivery" } } } } } }