Which web-data tool should I use?
A use-case guide built on the published prices we collected (2026-10-16). Tables are generated from the data; the shortlists are our reading of vendor documentation, not benchmarks.
Quick decision list
- Ingesting docs or sites for RAG: start with the extraction tools above and test token or credit use on 100 real pages.
- Blocked by anti-bot: trial two unblocker APIs on your target domains and compare success rate first, price second.
- Agent needs fresh answers: use a search API, not a scraper.
- Need to log in or click: use browser infrastructure.
- Tiny volume: use the free tiers listed on each vendor page; several are recurring.
RAG and LLM ingestion
You need clean text (usually markdown) from many pages, often whole sites, then fresh answers at query time. Price is driven by how many pages you ingest and whether you need JavaScript rendering or LLM extraction.
Shortlist
- Firecrawl: credit-based scrape, crawl and search from one balance, built for this job. Watch extra credits for JSON extraction.
- Jina Reader: token-priced reader; our per-page figure rests on an assumption, so measure your own token counts.
- Crawl4AI: open source and self-hosted, so no vendor fees, but you pay for infrastructure and proxies yourself.
- For query-time retrieval rather than bulk ingestion: Tavily, Exa and Brave Search API.
| # | Vendor | Plan / rate | Price per 1,000 | Monthly spend needed | Billing |
|---|---|---|---|---|---|
| 1 | Firecrawl | Standard | $0.83 per 1,000 | $83/mo | annual |
| 2 | Exa | Contents (per content type) | $1.00 per 1,000 | none | payg |
| 3 | Tavily | Pay as you go | $1.60 per 1,000 | none | payg |
| # | Vendor | Plan / rate | Price per 1,000 | Monthly spend needed | Billing |
|---|---|---|---|---|---|
| 1 | Jina Reader | $50 pack (s.jina.ai minimum) | $0.50 per 1,000 | none | prepaid |
| 2 | Browserbase | Parallel Fast | $1.00 per 1,000 | $20/mo | payg |
| 3 | Firecrawl | Standard | $1.66 per 1,000 | $83/mo | annual |
| 4 | Exa | Instant search | $4.00 per 1,000 | none | payg |
| 5 | Brave Search API | Search | $5.00 per 1,000 | none | payg |
| 6 | Hyperbrowser | All plans | $5.00 per 1,000 | none | payg |
Head-to-head: Firecrawl vs Crawl4AI, Firecrawl vs Jina Reader, Tavily vs Exa, Tavily vs Firecrawl.
Scraping protected (anti-bot) sites
Sites behind Cloudflare, DataDome and similar defences need residential IPs, fingerprinting and retries. Vendors price this very differently (per request, per GB, or per credit multiplier), so we compare only where a per-request price is published. Success rate matters more than price here and we have not measured it.
Shortlist
- Zyte, Scrapfly, ScraperAPI and ZenRows publish per-request or per-credit pricing for protected pages.
- Bright Data and Oxylabs sell large proxy and unblocker networks; compare them with Bright Data vs Oxylabs.
- Apify is a marketplace of ready-made scrapers; its prices are per Actor, so no per-page figure is possible.
| # | Vendor | Plan / rate | Price per 1,000 | Monthly spend needed | Billing |
|---|---|---|---|---|---|
| 1 | Zyte | $100/mo commitment - Tier 5 HTTP | $0.95 per 1,000 | $100/mo | monthly |
| 2 | Apify | Free/Starter (Unblocker) component only | $1.50 per 1,000 | none | payg |
| 3 | Bright Data | Unlocker PAYG | $1.50 per 1,000 | none | payg |
| 4 | ScrapingBee | Startup | $2.48 per 1,000 | $99/mo | monthly |
| 5 | Scrapfly | Pro | $2.50 per 1,000 | $100/mo | monthly |
| 6 | ScraperAPI | Hobby | $5.39 per 1,000 | $49/mo | monthly |
| # | Vendor | Plan / rate | Price per GB | Monthly spend needed | Billing |
|---|---|---|---|---|---|
| 1 | Oxylabs | Flexible | $4.00 per GB | $500/mo | monthly |
| 2 | Apify | Business | $7.00 per GB | $999/mo | monthly |
| 3 | Bright Data | PAYG (list) | $8.00 per GB | none | payg |
Run a trial on your own target sites before committing. A cheaper per-request price that fails 30% of the time is not cheaper.
Search results (SERP) for agents
Two different products share this label: classic SERP scrapers that return a search engine's result page as JSON, and AI-oriented search APIs that return ranked, cleaned results. Check which one your agent needs.
Shortlist
- SerpApi: Google and other engines as structured JSON, plans scale by monthly searches.
- Brave Search API: independent index and pay-per-request pricing.
- Tavily and Exa: AI-oriented search with agent integrations.
- Bright Data: SERP API inside a larger proxy platform.
| # | Vendor | Plan / rate | Price per 1,000 | Monthly spend needed | Billing |
|---|---|---|---|---|---|
| 1 | Crawl4AI | Cloud Builder | $0.792 per 1,000 | $99/mo | monthly |
| 2 | Oxylabs | Starter - /Search | $1.00 per 1,000 | $30/mo | monthly |
| 3 | Bright Data | SERP PAYG | $1.50 per 1,000 | none | payg |
| 4 | Apify | Free/Starter (SERP proxy) component only | $2.50 per 1,000 | none | payg |
| 5 | ScraperAPI | Hobby (Google = 25 credits) | $12.25 per 1,000 | $49/mo | monthly |
| 6 | SerpApi | Developer | $15.00 per 1,000 | $75/mo | monthly |
| # | Vendor | Plan / rate | Price per 1,000 | Monthly spend needed | Billing |
|---|---|---|---|---|---|
| 1 | Jina Reader | $50 pack (s.jina.ai minimum) | $0.50 per 1,000 | none | prepaid |
| 2 | Browserbase | Parallel Fast | $1.00 per 1,000 | $20/mo | payg |
| 3 | Firecrawl | Standard | $1.66 per 1,000 | $83/mo | annual |
| 4 | Exa | Instant search | $4.00 per 1,000 | none | payg |
| 5 | Brave Search API | Search | $5.00 per 1,000 | none | payg |
| 6 | Hyperbrowser | All plans | $5.00 per 1,000 | none | payg |
Head-to-head: SerpApi vs Bright Data, Tavily vs Brave Search API.
Browser automation for agents
Agents that click, log in or fill forms need a real browser in the cloud. Pricing is usually per browser hour plus a monthly platform fee, with extra charges for proxies and captcha solving that we list but do not fold in.
Shortlist
- Browserbase, Hyperbrowser and Steel sell managed headless browsers.
- Compare: Browserbase vs Hyperbrowser, Browserbase vs Steel.
| # | Vendor | Plan / rate | Price per hour | Monthly spend needed | Billing |
|---|---|---|---|---|---|
| 1 | Hyperbrowser | All plans | $0.10 per hour | none | payg |
| 2 | Steel | Launch | $0.10 per hour | none | payg |
| 3 | Browserbase | Startup (included hours) | $0.198 per hour | $99/mo | monthly |
Browser hours are only one cost. Concurrency limits, session length and proxy bandwidth often decide the real bill.
Cheapest for plain page fetches
If you only need HTML from ordinary pages, fetch cost is small and rate limits matter more.
| # | Vendor | Plan / rate | Price per 1,000 | Monthly spend needed | Billing |
|---|---|---|---|---|---|
| 1 | ScrapingBee | Startup | $0.099 per 1,000 | $99/mo | monthly |
| 2 | Scrapfly | Pro | $0.10 per 1,000 | $100/mo | monthly |
| 3 | Zyte | $100/mo commitment - Tier 1 HTTP | $0.10 per 1,000 | $100/mo | monthly |
| 4 | ZenRows | Launch | $0.232 per 1,000 | $58/mo | annual |
| 5 | Crawl4AI | Cloud Builder | $0.396 per 1,000 | $99/mo | monthly |
| 6 | ScraperAPI | Hobby | $0.441 per 1,000 | $44.1/mo | annual |
| 7 | Browserbase | Startup (Fetch API) | $0.50 per 1,000 | $99/mo | payg |
| 8 | Firecrawl | Standard | $0.83 per 1,000 | $83/mo | annual |
| # | Vendor | Plan / rate | Price per 1,000 | Monthly spend needed | Billing |
|---|---|---|---|---|---|
| 1 | ScrapingBee | Startup | $0.495 per 1,000 | $99/mo | monthly |
| 2 | Scrapfly | Pro | $0.50 per 1,000 | $100/mo | monthly |
| 3 | Zyte | $100/mo commitment - Tier 1 rendered | $0.75 per 1,000 | $100/mo | monthly |
| 4 | Crawl4AI | Cloud Builder | $0.792 per 1,000 | $99/mo | monthly |
| 5 | ZenRows | Launch | $1.16 per 1,000 | $58/mo | annual |
| 6 | Oxylabs | Starter - custom domain JS | $1.50 per 1,000 | $30/mo | monthly |