API reference

API documentation

Every endpoint and parameter, with cURL, JavaScript and Python examples. Base URL: https://crawlwisedata.com

Quickstart: zero to first call (and first webhook)

Five minutes to a successful keyed call and a scheduled monitor that POSTs a signed change event.

  1. Try it without a key (free demo, 20 calls per day per IP):
    Shell
    curl "https://crawlwisedata.com/v1/techstack?url=wordpress.org"

    Or use the free browser tools: tech stack lookup · sitemap & feed checker. Compare: vs Visualping · vs BuiltWith · for AI agents.

  2. Get a key: request a free trial key (1,000 one-time credits, no card) with the form below or POST /v1/trial, or buy a credit pack or monthly plan on the pricing page. Either way the key is shown once. Copy it.
    Shell
    curl -X POST "https://crawlwisedata.com/v1/trial"   -H "Content-Type: application/json"   -d '{"email":"you@company.com","acceptTerms":true}'

    Get a free trial key

    1,000 free credits, one time (about 33 tech-stack lookups or 100 feed reads). No card. One key per email every 30 days; per network up to 5 requests an hour and 10 a day. We never email you.

    Trial limits: 20 requests/min, 5 monitors. If a call costs more than your balance, it's refused and you're charged nothing. Disposable email addresses are refused. We keep only a salted hash of your email (Privacy). Agents: POST /v1/trial with {"email": "…", "acceptTerms": true} (or accept later with POST /v1/terms/accept).

  3. One successful keyed call (copyable):
    export CRAWLWISE_KEY=cw_test_...
    curl -G "https://crawlwisedata.com/v1/techstack"   -H "Authorization: Bearer $CRAWLWISE_KEY"   -d url=wordpress.org
    const res = await fetch('https://crawlwisedata.com/v1/techstack?url=wordpress.org', {
      headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}` },
    });
    const json = await res.json();
    if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
    console.log(json.data.technologyNames, json.charged);
    import os, requests
    r = requests.get(
        'https://crawlwisedata.com/v1/techstack',
        params={'url': 'wordpress.org'},
        headers={'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"},
        timeout=60,
    )
    data = r.json()
    if not data.get('ok'):
        raise RuntimeError(data.get('error'))
    print(data['data']['technologyNames'], data['charged'])
  4. Create a monitor + webhook (Crawlwise runs the schedule; you receive signed POSTs when something changes):
    curl -X POST "https://crawlwisedata.com/v1/monitors"   -H "Authorization: Bearer $CRAWLWISE_KEY"   -H "Content-Type: application/json"   -d '{
        "endpoint": "sitemap",
        "name": "competitor",
        "url": "https://example.com",
        "webhookUrl": "https://your.app/hooks/crawlwise",
        "schedule": true,
        "intervalSec": 900
      }'
    
    # Fetch the HMAC secret used in X-Crawlwise-Signature
    curl -H "Authorization: Bearer $CRAWLWISE_KEY"   "https://crawlwisedata.com/v1/webhooks/secret"
    const headers = {
      Authorization: `Bearer ${process.env.CRAWLWISE_KEY}`,
      'Content-Type': 'application/json',
    };
    const mon = await fetch('https://crawlwisedata.com/v1/monitors', {
      method: 'POST', headers,
      body: JSON.stringify({
        endpoint: 'sitemap', name: 'competitor',
        url: 'https://example.com',
        webhookUrl: 'https://your.app/hooks/crawlwise',
        schedule: true, intervalSec: 900,
      }),
    }).then((r) => r.json());
    const secret = await fetch('https://crawlwisedata.com/v1/webhooks/secret', {
      headers: { Authorization: headers.Authorization },
    }).then((r) => r.json());
    console.log(mon, secret.secret);
    import os, requests
    h = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
    mon = requests.post('https://crawlwisedata.com/v1/monitors', headers={**h, 'Content-Type': 'application/json'}, json={
        'endpoint': 'sitemap', 'name': 'competitor',
        'url': 'https://example.com',
        'webhookUrl': 'https://your.app/hooks/crawlwise',
        'schedule': True, 'intervalSec': 900,
    }, timeout=60).json()
    secret = requests.get('https://crawlwisedata.com/v1/webhooks/secret', headers=h, timeout=60).json()
    print(mon, secret['secret'])

    Verify deliveries with the Node/Python snippets under Monitors & change alerts (X-Crawlwise-Signature).

  5. See it work right away: POST /v1/monitors/test sends a free signed sample event to your webhook, and the built-in demo target produces a real change every 5 minutes. Walkthrough: See a change during your trial.
  6. Check your balance any time with GET /v1/account or in the dashboard.
  7. Agents: import OpenAPI, or run the local crawlwise-mcp MCP server (see AI agents).

See a change during your trial

You don't need a site that happens to change, or to wait for a schedule. With a trial key (1,000 credits) the whole loop costs about 50 credits:

  1. Create a monitor on the built-in demo target (/demo-target/feed.xml gains an item every 5 minutes; /demo-target/sitemap.xml adds, removes and updates a URL every 5 minutes):
    Shell
    curl -X POST "https://crawlwisedata.com/v1/monitors"   -H "Authorization: Bearer $CRAWLWISE_KEY" -H "Content-Type: application/json"   -d '{"endpoint":"feed","name":"demo","url":"https://crawlwisedata.com/demo-target/feed.xml",
           "webhookUrl":"https://your.app/hooks/crawlwise","schedule":true}'
  2. Send a test event (free, 10/hour): a signed monitor.test POST to your webhookUrl, signed exactly like real events. The response shows the exact body and signature sent, so you can check your verifier.
    Shell
    curl -X POST "https://crawlwisedata.com/v1/monitors/test"   -H "Authorization: Bearer $CRAWLWISE_KEY" -H "Content-Type: application/json"   -d '{"endpoint":"feed","name":"demo"}'
  3. Take the baseline now (10 credits for a feed; a sitemap baseline is a flat 10 credits for any site size):
    Shell
    curl -X POST "https://crawlwisedata.com/v1/monitors/run"   -H "Authorization: Bearer $CRAWLWISE_KEY" -H "Content-Type: application/json"   -d '{"endpoint":"feed","name":"demo"}'
  4. After the minimum interval (the same on every plan; a 429 includes Retry-After), run it again. The demo feed has new items, so the response shows "changed": true and Crawlwise POSTs a real signed monitor.change event to your webhook (10 credits). Scheduled monitors do the same on their own.

Your own sites work the same way. For a big sitemap, set max high enough for the whole site (up to 10,000) on the baseline: it is still a flat 10 credits, and later checks bill only for changed URLs.

Authentication

Send your key as Authorization: Bearer cw_live_... (or X-API-Key: cw_live_...). Keys start with cw_live_ (live) or cw_test_ (test mode); older wd_live_ keys keep working. Keys are 40 characters, and we store only a SHA-256 hash, so a key can never be shown again. If you lose a key, recover a new one; if it leaks, rotate it (POST /v1/keys/rotate). Without a key, data endpoints run in demo mode with smaller limits. Dashboard sessions (cws_..., 30 minutes) work for account endpoints only. Never put your key in public front-end code: keyed endpoints don't send CORS headers, on purpose.

Responses

All responses are JSON. Success: { "ok": true, "endpoint", "mode": "paid"|"demo", "charged": {credits, usd}, "balance": {credits, usd} | null, "data": {...} }. Response headers: X-Credits-Charged, X-Credits-Remaining, X-RateLimit-Limit, X-RateLimit-Remaining; Retry-After on 429.

Errors and status codes

Every error has the same shape and is never billed:

JSON
{ "ok": false, "error": { "code": "insufficient_credits", "message": "Insufficient credits", "billable": false, ... }, "charged": { "credits": 0, "usd": 0 } }
codeHTTPMeaning / what to do
invalid_params400A parameter is missing, unknown or out of range. Fix the request.
invalid_url400The url parameter is not a valid URL.
unsupported_url400Not allowed: non-http(s) scheme, non-default port, credentials in the URL, internal hostname, or a private/reserved IP (also after DNS resolution and redirects).
invalid_key401Missing, malformed, revoked or unknown API key/session.
auth_required401This endpoint needs a key (or the demo is disabled).
insufficient_credits402Balance too low for this call. One-off GET /v1/sitemap needs 10 + max credits up front (lower max). Monitor checks (scheduled, check-now and ?monitor=) reserve only the cheapest possible check (2 credits; 5 for techstack; 10 for a first snapshot) and are refused only below it; the real cost is charged when the check finishes, from the balance available then, and a sitemap change larger than that balance delivers the URLs it covers and carries the rest over. Includes needed, available and topUp links; nothing is charged.
account_suspended403Negative balance (top up), open payment dispute, or suspended for abuse (contact support).
robots_disallowed403The site's robots.txt disallows this URL for us (or robots.txt returned a 5xx error). We never fetch it.
not_found404Unknown route or object.
method_not_allowed405Wrong HTTP method.
trial_not_available409POST /v1/trial: email already used in the last 30 days (details.reason). Network limits answer 429 rate_limited with Retry-After (at most 24 h). Use the free demo or buy a pack.
signup_not_available409POST /v1/signup: a free account was already created for this email in the last 30 days (details.reason). Per-network signup limits (hour/day, Cloudflare WARP = one network) return 429 with Retry-After of at most 24 hours. Sign in with your key or contact support.
payload_too_large413Request body too large.
no_data_found422The target was reachable but had no feed/sitemap we could parse.
target_error422The target failed: DNS error, connection refused, TLS error, HTTP ≥ 400 (often bot blocking), response too large, or too many redirects.
rate_limited429Per-minute, concurrency, monitor-frequency, demo or form limit reached. Respect Retry-After.
internal_error500Our fault. You were not charged. Retry with backoff.
feature_unavailable503A dependency (e.g. payments) is not configured or temporarily unreachable.
timeout504The target did not answer within the time limit.

Limits

Free trialPay as you go (packs)Starter
Requests per minute per account2060120
Concurrent requests235
Monitors5100500

All plans: max 25 s per call, 15 s per outbound request, 5 redirects, page ≤ 5 MB, feed ≤ 10 MB and 200 items, sitemap file ≤ 15 MB (after decompression), ≤ 50 sitemap files, ≤ 60 MB total and ≤ 10,000 URLs per call, 3 active keys. Only http(s) on ports 80/443 to public IP addresses. robots.txt is always respected.

Demo (no key): 20 calls/day per IP, 60/day per /24 (IPv4) or /64 (IPv6) network, 500/day in total; feed ≤ 15 items, sitemap ≤ 75 URLs, no monitors. Demo attempts count even when they fail. On 429, Retry-After is seconds until 00:00 UTC.

Costs

1 credit = $0.0001. Tech stack 30 credits, feed 10 credits, sitemap 10 + 1/URL. Monitor re-checks with no change: 2 credits (feed/sitemap) or 5 (techstack). When a change is found, the full endpoint price applies (sitemap bills per changed URL): delivering a sitemap change needs at least 11 credits (10 base + 1 per URL); a check that finds no change costs 2. If your balance covers only part of a change, the URLs it pays for are delivered and the rest carry over; delivering carried-over URLs later costs 1 credit each (no second base fee). A refusal is free and the monitor shows change detected, top up to fetch remaining N URLs. Sitemap monitor baseline (first snapshot): flat 10 credits for any site size (50/day per account, 10 on a trial; then per URL). Test events (POST /v1/monitors/test): free. Failed, blocked, timed-out, robots-disallowed and invalid calls cost nothing. Credits are reserved when a call starts and only the actual cost is deducted when it succeeds. Packs: $10 = 100,000, $25 = 262,500. Monthly: Starter $19 = 250,000/month. Details: pricing.

  • pack_10: 100,000 credits for $10, one-time. Credits never expire while your account is active. A sitemap change costs at least 11 credits; a check with no change costs 2.
  • pack_25: 262,500 credits for $25, one-time (5% bonus). Credits never expire while your account is active. A sitemap change costs at least 11 credits; a check with no change costs 2.
  • starter: 250,000 credits every month for $19, about 43 monitors. Scheduled checks, usually within an hour, plus check-now on demand. Starter credits carry over for one billing period, then expire.

Monitors & change alerts

We run them for you. POST /v1/monitors with schedule: true and an https webhookUrl. Crawlwise checks the target on its schedule (scheduled checks, usually within an hour, plus check-now on demand) and POSTs a signed JSON event when something changes (new feed items, sitemap add/update/remove, tech stack add/remove). Noop checks cost 2 credits (5 for tech). Get your signing secret from GET /v1/webhooks/secret.

Verify deliveries (Node):

JavaScript
import { createHmac, timingSafeEqual } from 'node:crypto';
function verify(secret, rawBody, header) {
  const m = /^t=(\d+),v1=([0-9a-f]{64})$/i.exec(header || '');
  if (!m) return false;
  const expected = createHmac('sha256', secret).update(`${m[1]}.${rawBody}`).digest();
  const got = Buffer.from(m[2], 'hex');
  return got.length === expected.length && timingSafeEqual(got, expected);
}

Python:

Python
import hmac, hashlib, re
def verify(secret, raw_body, header):
    m = re.match(r'^t=(\d+),v1=([0-9a-f]{64})$', header or '', re.I)
    if not m: return False
    expected = hmac.new(secret.encode(), f'{m[1]}.{raw_body}'.encode(), hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, m[2].lower())

Retries. A delivery gets one attempt with a ~5 s timeout; redirects are not followed. If it fails (non-2xx, timeout, network error), we retry on later scheduler ticks with backoff (about 1 min, 5 min, 30 min, 2 h, 6 h; 6 attempts in total). A retry resends the same event: the same body, the same X-Crawlwise-Event id and the same X-Crawlwise-Delivery id, re-signed with a fresh timestamp (t= in X-Crawlwise-Signature, also in X-Crawlwise-Timestamp) on every attempt, so a timestamp tolerance check (we suggest rejecting t= older than 5 minutes) still passes on late retries. Retries never re-run the check and never charge again. Deduplicate on X-Crawlwise-Delivery (or the event id).

You can still poll yourself with ?monitor=NAME on feed/sitemap (same noop pricing), or call POST /v1/monitors/run to check a monitor now and get the webhook if it changed. POST /v1/monitors/test sends a free signed sample event (type monitor.test, header X-Crawlwise-Test: 1). Delete with DELETE /v1/monitors.

AI agents & OpenAPI

Machine-readable surface for agents and codegen:

  • /openapi.json (also YAML) — OpenAPI 3 for every public endpoint plus the monitor.change and monitor.test webhook event schemas.
  • /llms.txt and /llms-full.txt — short indexes for LLM crawlers.
  • Free trial key for agents: POST /v1/trial with {"email":"…"} returns a key with 1,000 one-time credits (one per email every 30 days; per-network hourly and daily limits for shared egress). Set it as CRAWLWISE_API_KEY.
  • MCP: local stdio package crawlwise-mcp (tools: techstack, feed, sitemap, list_monitors, create_monitor, test_monitor_webhook, run_monitor, get_account). Install it with npx crawlwise-mcp and add it to Claude Desktop / Cursor. Auth: env CRAWLWISE_API_KEY. No AI runs inside Crawlwise — thin HTTP wrapper.

Data endpoints

GET /v1/techstack

Detects the technologies used by one web page (CMS, e-commerce platform, frameworks, analytics, CDN, server, and email/DNS providers from DNS records). Uses 7,600+ open-source fingerprints and reports which signal matched. SaaS domain-verification TXT records (openai-domain-verification=…, MS=…, stripe-verification=…) only prove someone set up an account for the domain, so they are NOT reported as technologies: they appear in data.verifications with confidence 0 and never in technologies, technologyNames or categories. SPF includes (v=spf1 include:sendgrid.net) still count as mail senders.

Auth: key (or none for the free demo)Cost: 30 credits ($0.003) per successfully analyzed page
ParameterRulesMeaning
urlrequiredPage to analyze. A bare domain means https://domain/.
dnsoptional, default 10 skips the MX/TXT/NS/SOA/CNAME lookups.
minConfidenceoptional, 0-100, default 0Drop detections below this confidence.
impliedoptional, default 10 hides technologies that are only implied by others (e.g. PHP implied by WordPress).
curl \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  "https://crawlwisedata.com/v1/techstack?url=wordpress.org&minConfidence=50"
const res = await fetch('https://crawlwisedata.com/v1/techstack?url=wordpress.org&minConfidence=50', {
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}` },
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.get('https://crawlwisedata.com/v1/techstack', params={'url':'wordpress.org','minConfidence':'50'}, headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{
  "ok": true, "endpoint": "techstack", "mode": "paid",
  "charged": { "credits": 30, "usd": 0.003 },
  "balance": { "credits": 99970, "usd": 9.997 },
  "data": {
    "url": "https://wordpress.org/", "domain": "wordpress.org", "statusCode": 200, "title": "Blog Tool, Publishing Platform, and CMS",
    "technologyCount": 6, "technologyNames": ["WordPress", "Nginx", "PHP", "MySQL"],
    "technologies": [{ "name": "WordPress", "version": null, "confidence": 100, "categories": ["CMS", "Blogs"], "implied": false, "detectedBy": ["meta", "html"] }],
    "categories": { "CMS": ["WordPress"] },
    "verifications": [{ "name": "OpenAI", "confidence": 0, "signal": "dns-txt-domain-verification", "record": "TXT", "categories": ["AI"], "alsoDetectedAsTechnology": false }],
    "verificationNote": "verifications = SaaS domain-verification DNS TXT records. They show an account was once set up for this domain, not that the site uses the product, so they are excluded from technologies (confidence 0)."
  }
}

GET /v1/feed

Reads an RSS 2.0, Atom, RDF or JSON Feed and returns one normalized schema. Pass a website URL and the feed is discovered from its <link rel="alternate"> tag. With monitor=NAME each call returns only items you have not seen before under that name.

Auth: key (or none for the free demo)Cost: 10 credits ($0.001) per successful feed read (up to 200 items)
ParameterRulesMeaning
urlrequiredFeed URL or website URL.
maxoptional, 1-200 (demo: 1-15), default 50 (demo 20)Maximum items returned.
includeoptional, repeatable, max 10Keep items whose title/summary contains any of these keywords (case-insensitive).
excludeoptional, repeatable, max 10Drop items containing any of these keywords.
publishedAfteroptionalA date (2026-09-01) or relative period (7 days, 12 hours).
contentoptional, default 01 includes full item HTML (contentHtml).
monitoroptional (keys only), 1-40 of A-Z a-z 0-9 _ -New-items-only mode. The first call is the baseline and returns everything currently in the feed.
curl \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  "https://crawlwisedata.com/v1/feed?url=https://blog.example.com&monitor=my-list&include=release"
const res = await fetch('https://crawlwisedata.com/v1/feed?url=https://blog.example.com&monitor=my-list&include=release', {
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}` },
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.get('https://crawlwisedata.com/v1/feed', params={'url':'https://blog.example.com','monitor':'my-list','include':'release'}, headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{
  "ok": true, "endpoint": "feed", "mode": "paid",
  "charged": { "credits": 10, "usd": 0.001 },
  "balance": { "credits": 99960, "usd": 9.996 },
  "data": {
    "feedUrl": "https://blog.example.com/feed.xml", "discoveredFrom": "https://blog.example.com/", "title": "Example Blog", "itemsInFeed": 20,
    "items": [{ "itemId": "https://blog.example.com/p/1", "title": "Release 2.0", "link": "https://blog.example.com/p/1", "publishedAt": "2026-09-27T12:00:00.000Z", "author": "Ann", "categories": ["news"], "summary": "...", "enclosures": [] }],
    "monitor": { "name": "my-list", "baseline": false, "newItems": 1 }
  }
}

GET /v1/sitemap

Extracts every URL a site lists in its XML, gzipped or plain-text sitemaps (found through robots.txt Sitemap lines or common locations), with lastmod, changefreq, priority, image/video counts and hreflang alternates. With monitor=NAME you get only new, updated and removed URLs since the last call.

Auth: key (or none for the free demo)Cost: 10 credits per call + 1 credit per URL returned ($0.001 + $0.10 per 1,000 URLs). The maximum possible cost (10 + max) is reserved before the call and only the actual cost is charged. Monitor baseline (first call with a new monitor name): flat 10 credits for any site size, and only that is reserved.
ParameterRulesMeaning
urlrequiredSite URL or a sitemap URL.
maxoptional, 1-10,000 (demo: 1-75), default 1,000 (demo 100)Maximum URLs returned.
includeoptional, repeatable, max 10, regex ≤200 charsKeep URLs matching any regex (case-insensitive). Backreferences and nested quantifiers are rejected.
excludeoptional, repeatableDrop URLs matching any regex.
lastmodAfteroptionalKeep URLs whose lastmod is on/after this date.
monitoroptional (keys only)Change-detection mode. The first call (baseline) costs a flat 10 credits for any site size and returns counts plus a 20-URL sample (monitor.urlsInSnapshot); 50 flat baselines per account per day (10 on a trial), then per-URL pricing. Set max high enough for the whole site on the baseline: it costs the same. If a result is truncated the snapshot is not updated and "removed" is not reported.
curl \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  "https://crawlwisedata.com/v1/sitemap?url=example.com&max=5000&monitor=competitor"
const res = await fetch('https://crawlwisedata.com/v1/sitemap?url=example.com&max=5000&monitor=competitor', {
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}` },
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.get('https://crawlwisedata.com/v1/sitemap', params={'url':'example.com','max':'5000','monitor':'competitor'}, headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{
  "ok": true, "endpoint": "sitemap", "mode": "paid",
  "charged": { "credits": 13, "usd": 0.0013 },
  "data": {
    "site": "example.com", "discoveredVia": "robots.txt", "sitemapsFetched": 2, "truncated": false, "errors": [],
    "urls": [{ "url": "https://example.com/new-page", "lastmod": "2026-09-21", "changefreq": null, "priority": null, "imageCount": 0, "videoCount": 0, "alternates": [], "sitemapUrl": "https://example.com/sitemap.xml", "changeType": "new" }],
    "monitor": { "name": "competitor", "baseline": false, "counts": { "new": 1, "updated": 1, "removed": 1, "unchanged": 950 }, "snapshotSaved": true }
  }
}

Account

GET /v1/account

Balance (pack vs subscription credits, trial status: accountType "trial" | "standard" and a trial object with creditsGranted / creditsLeft for free trial keys, reserved, expiring grants), totals (purchased, consumed, clawed back, expired), monthly usage, the last 25 calls, plan and limits, subscription status, keys (prefix only), billing events, personal top-up/subscribe links and support info.

Auth: key or sessionCost: free
curl \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  "https://crawlwisedata.com/v1/account"
const res = await fetch('https://crawlwisedata.com/v1/account', {
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}` },
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.get('https://crawlwisedata.com/v1/account', headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{
  "ok": true, "accountId": "acct_...", "status": "active", "suspension": null,
  "balance": { "credits": 349970, "usd": 34.997, "packCredits": 100000, "subscriptionCredits": 249970, "reserved": 0, "available": 349970,
               "subscriptionGrants": [{ "credits": 249970, "grantedCredits": 250000, "expiresAt": "2026-11-28T00:00:00.000Z", "plan": "starter" }] },
  "totals": { "creditsPurchased": 100000, "subscriptionCreditsGranted": 250000, "creditsConsumed": 30, "requests": 2 },
  "plan": { "id": "starter", "name": "Starter", "limits": { "requestsPerMinute": 120, "concurrentRequests": 5, "maxMonitors": 500, "monitorMinIntervalSec": 900 } },
  "subscription": { "id": "sub_...", "status": "active", "plan": "starter", "currentPeriodEnd": "2026-10-28T00:00:00.000Z", "cancelAtPeriodEnd": false },
  "keys": [{ "id": "key_ab12cd34ef56", "prefix": "cw_live_AbCd…", "active": true }],
  "topUp": [{ "label": "$10 - 100,000 credits", "item": "pack_10", "url": "https://crawlwisedata.com/dashboard.html?buy=pack_10", "checkout": { "method": "POST", "path": "/v1/checkout", "body": { "item": "pack_10" } } }]
}

POST /v1/session

Exchanges an API key for a dashboard session token (cws_...) valid for 30 minutes. Sessions work for account endpoints only (not data endpoints) and are revoked when any key is revoked or rotated.

Auth: key onlyCost: free
curl -X POST \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  "https://crawlwisedata.com/v1/session"
const res = await fetch('https://crawlwisedata.com/v1/session', {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}` },
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.post('https://crawlwisedata.com/v1/session', headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "token": "cws_...", "expiresAt": "2026-09-28T20:30:00.000Z", "accountId": "acct_..." }

POST /v1/account/delete

Permanently deletes the account: cancels any subscription immediately (no further charges), revokes all keys, deletes monitors, usage history and the balance (remaining credits are forfeited). Payment records stay in Stripe for accounting.

Auth: key or sessionCost: free
ParameterRulesMeaning
confirmbody, requiredMust be exactly "DELETE".
curl -X POST \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  -H "Content-Type: application/json" \
  -d '{"confirm":"DELETE"}' \
  "https://crawlwisedata.com/v1/account/delete"
const res = await fetch('https://crawlwisedata.com/v1/account/delete', {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({"confirm":"DELETE"}),
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.post('https://crawlwisedata.com/v1/account/delete', json={'confirm':'DELETE'}, headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "deleted": "acct_...", "canceledSubscriptions": [], "note": "Account data deleted and all keys revoked. Payment records remain in Stripe for accounting." }

Free trial

POST /v1/trial

Issues a free trial API key with 1,000 one-time credits. The key is returned once in the response (we store only a hash). Emails are normalized (case, +tags and Gmail dots ignored); disposable or reserved domains and domains without mail servers are refused. Network limits: 5 requests per hour and 10 per day per network (an IPv4 address or an IPv6 /48; relay services such as Cloudflare WARP count as one network), 20 keys per day per /24, and 25 issued trial keys per UTC day site-wide (only issued keys count; refused requests never use a slot). A 429 says until when free trials are paused (the same time as Retry-After, never more than 24 hours) and offers the $10 pack (100,000 credits) with a checkout link (details.buy). Trial keys work on every endpoint with trial limits (20 req/min, 2 concurrent, 5 monitors, 1 active key) and can never go below zero: a call that needs more credits than are left returns 402. Buying a pack or plan from the dashboard keeps the key and ends the trial limits. Send Content-Type: application/json. Refusals: 409 trial_not_available (reason email), 400 invalid_params (reason disposable_email | domain_has_no_mail), 429 rate_limited (reason ip_attempts | ip_attempts_day | network | global). Trial keys must accept the Terms: send acceptTerms: true, or accept later with POST /v1/terms/accept using the key (until then 403 terms_required). GET /v1/trial returns the current rules.

Auth: none (one per email every 30 days; per network 5 attempts/hour and 10/day)Cost: free
ParameterRulesMeaning
emailbody, requiredYour email address. We never email you.
acceptTermsbody, recommendedtrue = you accept the Terms of Service and Privacy Policy (/terms.html). Without it the key is issued with termsRequired: true and calls answer 403 terms_required until you POST /v1/terms/accept {"acceptTerms":true} with the key.
curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"email":"you@company.com","acceptTerms":true}' \
  "https://crawlwisedata.com/v1/trial"
const res = await fetch('https://crawlwisedata.com/v1/trial', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({"email":"you@company.com","acceptTerms":true}),
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
r = requests.post('https://crawlwisedata.com/v1/trial', json={'email':'you@company.com','acceptTerms':True}, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{
  "ok": true, "trial": true, "accountId": "acct_...", "keyId": "key_...", "apiKey": "cw_test_...", "credits": 1000,
  "note": "Trial key: shown once, copy it now. We store only a hash.",
  "limits": { "requestsPerMinute": 20, "concurrentRequests": 2, "maxMonitors": 5, "activeKeys": 1 }
}

Keys

POST /v1/keys

Creates an additional key (max 3 active). The key is returned once.

Auth: key or sessionCost: free
curl -X POST \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  -H "Content-Type: application/json" \
  -d '{}' \
  "https://crawlwisedata.com/v1/keys"
const res = await fetch('https://crawlwisedata.com/v1/keys', {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({}),
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.post('https://crawlwisedata.com/v1/keys', json={}, headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "keyId": "key_ab12cd34ef56", "apiKey": "cw_test_...", "note": "Shown once. Store it securely." }

POST /v1/keys/rotate

Issues a new key and revokes ALL other keys and sessions immediately. Use this if a key may be compromised.

Auth: key or sessionCost: free
curl -X POST \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  -H "Content-Type: application/json" \
  -d '{}' \
  "https://crawlwisedata.com/v1/keys/rotate"
const res = await fetch('https://crawlwisedata.com/v1/keys/rotate', {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({}),
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.post('https://crawlwisedata.com/v1/keys/rotate', json={}, headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "keyId": "key_...", "apiKey": "cw_test_...", "note": "Shown once. All previous keys and dashboard sessions were revoked." }

POST /v1/keys/revoke

Disables one key ({"keyId":"key_..."}) or all keys ({"all":true}). Revocation is immediate. After revoking all keys, use key recovery to get a new one.

Auth: key or sessionCost: free
ParameterRulesMeaning
keyIdbody, orKey id from GET /v1/account.
allbodytrue to disable every key.
curl -X POST \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  -H "Content-Type: application/json" \
  -d '{"keyId":"key_ab12cd34ef56"}' \
  "https://crawlwisedata.com/v1/keys/revoke"
const res = await fetch('https://crawlwisedata.com/v1/keys/revoke', {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({"keyId":"key_ab12cd34ef56"}),
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.post('https://crawlwisedata.com/v1/keys/revoke', json={'keyId':'key_ab12cd34ef56'}, headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "revoked": ["key_ab12cd34ef56"] }

Monitors

GET /v1/monitors

Lists your monitors (name, endpoint, target, schedule, webhook, last/next run).

Auth: key or sessionCost: free
curl \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  "https://crawlwisedata.com/v1/monitors"
const res = await fetch('https://crawlwisedata.com/v1/monitors', {
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}` },
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.get('https://crawlwisedata.com/v1/monitors', headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "count": 1, "monitors": [{ "endpoint": "feed", "name": "my-list", "target": "https://blog.example.com/feed.xml", "schedule": true, "enabled": true, "webhookUrl": "https://example.com/hooks", "intervalSec": 900, "lastRunAt": "2026-10-05T14:00:00.000Z", "nextRunAt": "2026-10-05T14:15:00.000Z" }], "scheduling": { "noopCredits": { "feed": 2, "sitemap": 2, "techstack": 5 } } }

POST /v1/monitors

Create or update a monitor (PATCH /v1/monitors updates an existing one; 404 if it does not exist). With schedule:true and an https webhookUrl, Crawlwise runs the check for you and POSTs a signed change event when something changes. endpoint may be feed, sitemap or techstack. intervalSec has the same minimum on every plan (900 seconds, 15 minutes); scheduled checks run usually within an hour, plus check-now on demand. Deprecated: sending `target` instead of `url` still works for one release; the response then carries a `warnings` entry and the headers `Deprecation: true` and `X-Crawlwise-Warning: 299 - "target is deprecated; use url"` (no standard `Warning` header). An intervalSec below 900 is not refused: it is raised to 900 and the response says so (a `warnings` entry with code `interval_clamped` and `X-Crawlwise-Warning: 299 - "intervalSec 300 clamped to 900"`). webhookUrl gets the same public-address check as url when the monitor is created or updated: private, loopback, link-local and cloud metadata addresses (also written as IPv4-mapped IPv6, octal or hex, or a hostname that resolves to one) are refused with 400 invalid_params (`field: webhookUrl`).

Auth: key or sessionCost: free to create; checks billed when they run
ParameterRulesMeaning
endpointbody, requiredfeed | sitemap | techstack
namebody, required1-40 of A-Z a-z 0-9 _ -
urlbody, requiredTarget site, feed or sitemap URL
webhookUrlbodyhttps URL that receives signed POSTs
schedulebodytrue = Crawlwise runs it (requires webhookUrl)
enabledbodyfalse pauses scheduled runs
intervalSecbodyDesired seconds between checks (minimum 900, the same on every plan)
maxbodyOptional max items/URLs for feed/sitemap
curl -X POST \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  -H "Content-Type: application/json" \
  -d '{"endpoint":"feed","name":"my-list","url":"https://blog.example.com","webhookUrl":"https://example.com/hooks/crawlwise","schedule":true,"intervalSec":900}' \
  "https://crawlwisedata.com/v1/monitors"
const res = await fetch('https://crawlwisedata.com/v1/monitors', {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({"endpoint":"feed","name":"my-list","url":"https://blog.example.com","webhookUrl":"https://example.com/hooks/crawlwise","schedule":true,"intervalSec":900}),
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.post('https://crawlwisedata.com/v1/monitors', json={'endpoint':'feed','name':'my-list','url':'https://blog.example.com','webhookUrl':'https://example.com/hooks/crawlwise','schedule':True,'intervalSec':900}, headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "created": true, "monitor": { "endpoint": "feed", "name": "my-list", "schedule": true, "webhookUrl": "https://example.com/hooks/crawlwise", "nextRunAt": "..." } }

DELETE /v1/monitors

Deletes a monitor and its stored snapshot, freeing a monitor slot. Webhook deliveries still waiting for a retry are cancelled (never sent); the response always includes cancelledDeliveries (0 when there were none). Retries also stop when the account is suspended.

Auth: key or sessionCost: free
ParameterRulesMeaning
endpointrequiredfeed, sitemap or techstack
namerequiredMonitor name.
curl -X DELETE \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  "https://crawlwisedata.com/v1/monitors?endpoint=feed&name=my-list"
const res = await fetch('https://crawlwisedata.com/v1/monitors?endpoint=feed&name=my-list', {
  method: 'DELETE',
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}` },
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.delete('https://crawlwisedata.com/v1/monitors', params={'endpoint':'feed','name':'my-list'}, headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "deleted": { "endpoint": "feed", "name": "my-list" }, "cancelledDeliveries": 0 }

POST /v1/monitors/test

Sends a signed sample event (type "monitor.test", test: true, synthetic change shaped like the monitor endpoint) to the monitor's webhookUrl right now, signed with your account secret exactly like real change events, plus header X-Crawlwise-Test: 1. One attempt, 8 s timeout, redirects are not followed, non-public addresses are refused. The response echoes the exact body and X-Crawlwise-Signature that were sent, so you can test your verifier.

Auth: key or sessionCost: free (10 per hour per account)
ParameterRulesMeaning
endpointbody, requiredfeed | sitemap | techstack
namebody, requiredMonitor name (must have a webhookUrl).
curl -X POST \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  -H "Content-Type: application/json" \
  -d '{"endpoint":"feed","name":"demo"}' \
  "https://crawlwisedata.com/v1/monitors/test"
const res = await fetch('https://crawlwisedata.com/v1/monitors/test', {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({"endpoint":"feed","name":"demo"}),
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.post('https://crawlwisedata.com/v1/monitors/test', json={'endpoint':'feed','name':'demo'}, headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{
  "ok": true, "test": true, "delivered": true, "status": 200, "eventId": "evt_test_...", "deliveryId": "deliv_...",
  "charged": { "credits": 0, "usd": 0 },
  "sent": { "headers": { "x-crawlwise-signature": "t=1791000000,v1=<hex>", "x-crawlwise-test": "1" }, "body": "{\"id\":\"evt_test_...\",\"type\":\"monitor.test\",\"test\":true, ...}" }
}

POST /v1/monitors/run

Checks a monitor immediately. If something changed since the last check and the monitor has a webhookUrl, a real signed "monitor.change" event is POSTed (one attempt). Works with or without schedule:true. Bound by the minimum monitor interval, the same on every plan (429 with Retry-After), and 30 runs per hour per account. Failed checks are free.

Auth: key or sessionCost: same as a scheduled check (baseline: feed 10, sitemap 10 flat, techstack 30; no change 2 / 5; change: full price)
ParameterRulesMeaning
endpointbody, requiredfeed | sitemap | techstack
namebody, requiredMonitor name.
curl -X POST \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  -H "Content-Type: application/json" \
  -d '{"endpoint":"feed","name":"demo"}' \
  "https://crawlwisedata.com/v1/monitors/run"
const res = await fetch('https://crawlwisedata.com/v1/monitors/run', {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({"endpoint":"feed","name":"demo"}),
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.post('https://crawlwisedata.com/v1/monitors/run', json={'endpoint':'feed','name':'demo'}, headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{
  "ok": true,
  "run": { "endpoint": "feed", "name": "demo", "ok": true, "baseline": false, "changed": true, "credits": 10, "status": "changed",
           "change": { "kind": "feed.new_items", "newItems": 2, "items": [ ... ] },
           "delivery": { "ok": true, "id": "deliv_...", "eventId": "evt_...", "status": 200, "error": null } },
  "charged": { "credits": 10, "usd": 0.001 }, "balance": { "credits": 978, "usd": 0.0978 },
  "note": "Change found; webhook delivered."
}

GET /v1/demo-target

A built-in target on this host that changes every 5 minutes: /demo-target/feed.xml gains one item, and /demo-target/sitemap.xml gains one URL, drops one and updates one lastmod. Point a monitor at it to see a real diff and webhook during a trial.

Auth: none (CORS enabled)Cost: free (checks on it are billed normally)
curl \
  "https://crawlwisedata.com/v1/demo-target"
const res = await fetch('https://crawlwisedata.com/v1/demo-target');
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
r = requests.get('https://crawlwisedata.com/v1/demo-target', timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "feed": "https://crawlwisedata.com/demo-target/feed.xml", "sitemap": "https://crawlwisedata.com/demo-target/sitemap.xml", "changesEverySec": 300 }

Webhooks

GET /v1/webhooks/secret

Returns the HMAC-SHA256 secret used to sign change-alert deliveries (header X-Crawlwise-Signature: t=<unix>,v1=<hex>).

Auth: key or sessionCost: free
curl \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  "https://crawlwisedata.com/v1/webhooks/secret"
const res = await fetch('https://crawlwisedata.com/v1/webhooks/secret', {
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}` },
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.get('https://crawlwisedata.com/v1/webhooks/secret', headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "secret": "<64 hex chars>", "header": "X-Crawlwise-Signature", "format": "t=<unix>,v1=<hex hmac-sha256 of `${t}.${rawBody}`>" }

POST /v1/webhooks/secret/rotate

Replaces the webhook signing secret. Update your verifier immediately.

Auth: key or sessionCost: free
curl -X POST \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  -H "Content-Type: application/json" \
  -d '{}' \
  "https://crawlwisedata.com/v1/webhooks/secret/rotate"
const res = await fetch('https://crawlwisedata.com/v1/webhooks/secret/rotate', {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({}),
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.post('https://crawlwisedata.com/v1/webhooks/secret/rotate', json={}, headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "secret": "<new 64 hex chars>" }

GET /v1/webhooks/deliveries

Recent webhook deliveries (status, latency, errors). Each delivery keeps an attempts history (last 10 attempts: time, HTTP status, error text). Bodies and signatures are never stored.

Auth: key or sessionCost: free
ParameterRulesMeaning
limitoptional, 1-100Max rows (default 25).
curl \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  "https://crawlwisedata.com/v1/webhooks/deliveries?limit=25"
const res = await fetch('https://crawlwisedata.com/v1/webhooks/deliveries?limit=25', {
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}` },
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.get('https://crawlwisedata.com/v1/webhooks/deliveries', params={'limit':'25'}, headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "count": 1, "deliveries": [{ "id": "deliv_...", "eventId": "evt_...", "ok": true, "status": 200, "attempt": 2, "at": "...", "attempts": [{ "at": "...", "status": 500, "error": "HTTP 500" }, { "at": "...", "status": 200, "error": null }] }] }

Billing

POST /v1/checkout

Creates a Stripe Checkout Session on our server for THIS account (bound by the account id we set in the session metadata), so the purchase is added to your existing balance. Only sessions created this way are credited automatically. Items: pack_10 ($10, 100,000 credits), pack_25 ($25, 262,500 credits), starter ($19/month, 250,000 credits/month). Subscriptions are refused if you already have one (change plan in the portal instead).

Auth: key or sessionCost: free (opens Stripe Checkout)
ParameterRulesMeaning
itembody, requiredpack_10 | pack_25 | starter
curl -X POST \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  -H "Content-Type: application/json" \
  -d '{"item":"pack_25"}' \
  "https://crawlwisedata.com/v1/checkout"
const res = await fetch('https://crawlwisedata.com/v1/checkout', {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({"item":"pack_25"}),
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.post('https://crawlwisedata.com/v1/checkout', json={'item':'pack_25'}, headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "url": "https://checkout.stripe.com/c/pay/cs_test_...", "sessionId": "cs_test_..." }

POST /v1/checkout/start

Sign-up fallback, only when no sign-up method is available (email codes off and no OAuth provider): creates an account with 0 credits and returns its API key once, plus the Checkout URL for the item. The key works only for checkout and account reads until the purchase completes, then it carries the paid credits. Limited to 30 attempts per hour per network (an IPv4 address or an IPv6 /48; relay services such as Cloudflare WARP count as one network) and 300 per hour site-wide, separately from POST /v1/trial; over the limit you get 429 rate_limited ("Too many checkout attempts from your network, try again in N minutes.") with Retry-After. The key is never put in a URL. Otherwise 409 signup_not_available (sign up first).

Auth: noneCost: free (opens Stripe Checkout)
ParameterRulesMeaning
itembody, requiredpack_10 | pack_25 | starter
acceptTermsbody, requiredtrue
curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"item":"pack_10","acceptTerms":true}' \
  "https://crawlwisedata.com/v1/checkout/start"
const res = await fetch('https://crawlwisedata.com/v1/checkout/start', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({"item":"pack_10","acceptTerms":true}),
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
r = requests.post('https://crawlwisedata.com/v1/checkout/start', json={'item':'pack_10','acceptTerms':True}, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "accountId": "acct_...", "apiKey": "cw_...", "credits": 0, "pendingPurchase": true, "keyNote": "This is your API key. It starts working as soon as your purchase completes. Save it now, it's shown once.", "checkoutUrl": "https://checkout.stripe.com/c/pay/cs_test_...", "sessionId": "cs_test_..." }

POST /v1/portal

Returns a Stripe Customer Portal link: change plan, cancel at period end, update card, download invoices.

Auth: key or sessionCost: free
curl -X POST \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  -H "Content-Type: application/json" \
  -d '{}' \
  "https://crawlwisedata.com/v1/portal"
const res = await fetch('https://crawlwisedata.com/v1/portal', {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({}),
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
headers = {'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"}
r = requests.post('https://crawlwisedata.com/v1/portal', json={}, headers=headers, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "url": "https://billing.stripe.com/p/session/..." }

Key recovery

POST /v1/recover/start

Starts key recovery for the email used at checkout. Returns a code and a Stripe portal login link and sets an HttpOnly cookie. Use the /recover.html page; it does this for you.

Auth: none (rate limited: 5 per hour per IP)Cost: free
ParameterRulesMeaning
emailbody, requiredCheckout email.
curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"email":"you@example.com"}' \
  "https://crawlwisedata.com/v1/recover/start"
const res = await fetch('https://crawlwisedata.com/v1/recover/start', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({"email":"you@example.com"}),
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
r = requests.post('https://crawlwisedata.com/v1/recover/start', json={'email':'you@example.com'}, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "code": "CW-7K2P-9QXM", "expiresAt": "...", "portalLoginUrl": "https://billing.stripe.com/p/login/...?prefilled_email=..." }

POST /v1/recover/status

Poll after updating your name in the portal. Returns state pending | verified (with a NEW key, once) | used | expired | none.

Auth: recovery cookie from /v1/recover/startCost: free
curl -X POST \
  -H "Content-Type: application/json" \
  -d '{}' \
  "https://crawlwisedata.com/v1/recover/status"
const res = await fetch('https://crawlwisedata.com/v1/recover/status', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({}),
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
r = requests.post('https://crawlwisedata.com/v1/recover/status', json={}, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "state": "verified", "accountId": "acct_...", "apiKey": "cw_test_...", "note": "All previous keys were revoked. This key is shown only once." }

Public

GET /v1/pricing

Machine-readable prices, packs, subscriptions, limits and refund policy.

Auth: none (CORS enabled)Cost: free
curl \
  "https://crawlwisedata.com/v1/pricing"
const res = await fetch('https://crawlwisedata.com/v1/pricing');
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
r = requests.get('https://crawlwisedata.com/v1/pricing', timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "creditUsd": 0.0001, "prices": { ... }, "packs": [ ... ], "subscriptions": [ ... ], "limits": { ... } }

GET /v1/health

Liveness + storage write/read probe. 200 when healthy, 503 when storage fails.

Auth: none (CORS enabled)Cost: free
curl \
  "https://crawlwisedata.com/v1/health"
const res = await fetch('https://crawlwisedata.com/v1/health');
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
r = requests.get('https://crawlwisedata.com/v1/health', timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "storage": "ok", "storageMs": 42, "time": "2026-09-28T19:00:00.000Z" }

GET /v1/status

Component status used by /status.html.

Auth: none (CORS enabled)Cost: free
curl \
  "https://crawlwisedata.com/v1/status"
const res = await fetch('https://crawlwisedata.com/v1/status');
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
r = requests.get('https://crawlwisedata.com/v1/status', timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "status": "operational", "checks": { "api": "ok", "storage": "ok", "payments": "ok", "checkout": "open", "demo": "enabled" } }

POST /v1/contact

Sends a message to support (stored securely; we reply by email if you give one).

Auth: none (5 messages per day per network)Cost: free
ParameterRulesMeaning
messagebody, required10-5000 characters.
emailbody, optionalWhere we should reply.
categorybody, optionalsupport | billing | abuse | opt-out | privacy | security | deletion | other
targetSitebody, optionalFor opt-out/abuse reports: the affected site.
curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"email":"you@example.com","category":"support","message":"My monitor returns no items, can you help?"}' \
  "https://crawlwisedata.com/v1/contact"
const res = await fetch('https://crawlwisedata.com/v1/contact', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({"email":"you@example.com","category":"support","message":"My monitor returns no items, can you help?"}),
});
const json = await res.json();
if (!json.ok) throw new Error(`${json.error?.code}: ${json.error?.message}`);
console.log(json);
import os, requests
r = requests.post('https://crawlwisedata.com/v1/contact', json={'email':'you@example.com','category':'support','message':'My monitor returns no items, can you help?'}, timeout=60)
data = r.json()
if not data.get('ok', r.ok):
    raise RuntimeError(data.get('error'))
print(data)
Example response
{ "ok": true, "id": "msg-2026-09-28T19:00:00.000Z-Ab12Cd", "note": "Thanks. We usually reply within 2 business days." }