Crawl: the most reliable web crawler APISee what’s new
Zenrows
Talk to sales Start free

Run large web data jobs reliably.

Send thousands. Scale to millions, job by job.

Process up to 100,000 URLs per job, asynchronously, protected and dynamic pages included, with queues, retries, per-job webhooks, and job observability built in.

What you'd ship without Batch

workers 0 32

Push more. Workers follow.

Workers spin up when the queue grows. They idle out when it drains. Add up to 10,000 URLs per call, as many calls as it takes to fill the job's 100,000-URL cap.

Everything else comes with it.

One signed webhook on completion. Retries on transient failure. Failed tasks stay visible. Live status endpoint. No glue code.

100,000 URLs a job. As many as needed.

Same workers. Same JSON contract per URL. Up to three active jobs per account (shared with Crawl).

Or just a list.

One URL per JSONL line. Estimating needs no key. Running uses the same API key as Fetch. The CLI's local safety policy caps how many URLs and estimated credits one run can submit, and stops a bigger run before it is sent; raise the caps with zenrows policy set.

{"url":"https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/"}
{"url":"https://www.scrapingcourse.com/ecommerce/product/adrienne-trek-jacket/"}
{"url":"https://www.scrapingcourse.com/ecommerce/product/aeon-capri/"}
$ npx -y @zenrows/cli batch estimate jobs.jsonl
valid jobs: 3/3
estimated credits, upper bound: 75
✓ Spec is valid.

$ npx -y @zenrows/cli batch create jobs.jsonl \
    --output markdown --follow
✓ Submitted job · status: completed
  3/3 completed · 3 successful · 0 failed

Push it. We drain it.

Start building free See pricing

A free plan with 5,000 credits every month.