CrawlwiseData
Use case

Website change alerts for AI agents

Agents shouldn’t poll the open web forever. Point Crawlwise at the sites that matter; we schedule checks and POST a signed webhook when a sitemap, feed, or tech stack actually changes. Deterministic Node.js — no browser farm, no AI at runtime on our side.

Why agents need change alerts (not more scraping)

Wake on signal

Spend agent turns when something moved — new docs page, new blog post, competitor CMS change — not on empty polls.

We own the boring parts

Scheduling, snapshots, dedupe, cheap noops, retries, HMAC signatures, robots.txt, SSRF protections.

Machine-readable by default

OpenAPI, llms.txt, JSON events, and a local MCP package (crawlwise-mcp) for Claude Desktop / Cursor — install it with npx crawlwise-mcp.

How it works

  1. Get a free trial key (POST /v1/trial) or buy credits — shown once as cw_live_….
  2. POST /v1/monitors with schedule: true, an endpoint (sitemap | feed | techstack), target URL, and https webhookUrl.
  3. Crawlwise runs scheduled checks, usually within an hour, plus check-now on demand. No change → cheap noop. Change → signed POST to your agent’s inbox or backend.
  4. Verify X-Crawlwise-Signature, then let the agent summarize, ticket, or act.

Pitch in one line: website change alerts, as signed webhooks — you pay for changes, not empty checks.

Agent-friendly surface area

AssetURL / packageUse it for
OpenAPI 3/openapi.jsonImport into agent tool runners
Agent index/llms.txt · /llms-full.txtShort capability brief for crawlers
Human docsdocs.htmlcURL / JS / Python + monitors howto
MCP (stdio)crawlwise-mcptechstack, feed, sitemap, monitors, account
TrialPOST /v1/trial1,000 credits, no card (limits apply)

Minimal monitor create

cURL
# After you have CRAWLWISE_API_KEY from /v1/trial or pricing
curl -X POST https://crawlwisedata.com/v1/monitors \
  -H "Authorization: Bearer $CRAWLWISE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "feed",
    "name": "competitor-blog",
    "url": "https://github.blog/feed/",
    "schedule": true,
    "webhookUrl": "https://your-app.example/hooks/crawlwise"
  }'

Full signature verification examples: Monitors howto. Compare positioning: vs Visualping (UI monitors) · vs BuiltWith (tech DB).

Try free — then wire an agent

Free trial key

1,000 one-time credits for monitors and full result sizes.

Get a free trial key

1,000 free credits, one time (about 33 tech-stack lookups or 100 feed reads). No card. One key per email every 30 days; per network up to 5 requests an hour and 10 a day. We never email you.

Trial limits: 20 requests/min, 5 monitors. If a call costs more than your balance, it's refused and you're charged nothing. Disposable email addresses are refused. We keep only a salted hash of your email (Privacy). Agents: POST /v1/trial with {"email": "…", "acceptTerms": true} (or accept later with POST /v1/terms/accept).

Pricing · Support · For site owners

FAQ

Does Crawlwise run an LLM on my targets?

No. Lookups and monitors are plain deterministic code. Your agent (optional) decides what to do with the event.

Can the agent call the API directly?

Yes. Prefer monitors + webhooks for continuous watch; use on-demand /v1/techstack, /v1/feed, /v1/sitemap for one-shots. MCP wraps the same API.

How do I keep webhooks safe?

Only accept HTTPS, verify HMAC with your account secret, and rotate via POST /v1/webhooks/secret/rotate if leaked.

Is this the same as Visualping or BuiltWith?

No. Visualping is visual/UI monitoring for humans; BuiltWith is a large tech database. Crawlwise is change-alert infrastructure for software. See the compare pages linked above.