Spider

Crawl / AI-focused platform T2 Scraping & Crawling /spider
Read the docs Provider site ↗

Spider is a concurrency-first crawling engine written in Rust that streams pages the instant they’re fetched, rendering JavaScript only when a page actually needs it — so it crawls fast and scales from a single script to a distributed fleet without code changes. Its core engine is fully open source under a permissive license, and the same code powers Spider Cloud, which layers on managed proxies, headless browsers and anti-bot handling. Transparent per-page pricing and clean markdown output make it a refreshingly developer-friendly choice for high-throughput crawling and AI data collection.

Entry price
~$0.03 / 1k (usage-based)
MCP
official (22 tools)
Latency
—

Choose if

You want fast crawling, clean markdown and transparent per-page pricing (anti-multiplier).

Avoid if

You need declarative actions (clicks no-op live) or forecastable usage-based billing.

Evals

How Spider performed on the curator-fleet web-access evals — the real measured numbers behind each composite, ranked best-first.

Crawlcomposite 8.54/10 #2 of 5
0.77
Coverage
2.9 s
Latency p50
$6.2e-05 / useful
Cost
0%
Errors
Fast recursive crawl.
Parse · PDF/doccomposite 6.99/10 #3 of 3
0.60
Text accuracy
2.0 s
Latency p50
$3.8e-05 / useful
Cost
0%
Errors
Returns raw bytes on scanned PDFs.
Scrapecomposite 6.73/10 #9 of 10
50%
Anti-bot bypass
9.6 / 10
Markdown clean
12.1 s
Latency p50
$0.00013 / success
Cost
0%
Errors
Screenshotcomposite 5.03/10 #7 of 8
0.50
Valid image
0.50
Full-page
8.4 s
Latency p50
$0.001 / call
Cost
38%
Errors
Act · declarativecomposite 1.5/10 #6 of 6
0.00
Task success
35.1 s
Latency p50
$0.001 / call
Cost
0%
Errors
Declarative clicks no-op live.
← All tools