Deep research agent
A deep research agent answers a question by planning searches, opening and reading many live web pages, and writing a report that cites each source. Zenrows handles the reading: Fetch returns every page as Markdown, protected and JavaScript-heavy sources included, and the MCP server lets Claude or your agent call it, so each claim keeps its URL.
Updated
{
"question": "How do three outdoor retailers handle returns of used gear?",
"sources_planned": 12,
"sources_read": 12,
"findings": [
{
"claim": "Retailer A accepts used items within 365 days for members.",
"source": "https://www.example-outdoor-a.com/help/returns",
"quote": "Members may return used gear within one year of purchase.",
"fetched_at": "2026-10-17T10:02:11Z"
},
{
"claim": "Retailer B gives store credit only for used items.",
"source": "https://www.example-outdoor-b.com/policies/returns",
"quote": "Used items are refunded as store credit.",
"fetched_at": "2026-10-17T10:02:14Z"
}
],
"unanswered": ["Retailer C's policy page does not cover worn footwear"]
}
quote; the agent writes the claims and keeps each URL and fetch time.What deep research agents investigate
A run loops: plan, find, read, check gaps, read again, write.
The agent can only state what it read.
| Job | Data you collect | Typical sources |
|---|---|---|
| Vendor and market research | Pricing, plans, features, release notes | Vendor pricing pages, docs, changelogs, |
| Company due diligence | Leadership, filings, press, litigation | Company sites, |
| Regulatory research | Rule text, effective dates, guidance | EUR-Lex, Federal Register, regulator websites |
| Literature scans | Abstracts, methods, results, citations | arXiv, |
| Technical investigations | Known issues, fixes, version differences | |
| Product comparisons | Specs, prices, reviews, return terms | Retailer product pages, manufacturer spec sheets |
Why research agents come back thin
Vague reports often start upstream: the model read snippets, not pages, or half its sources failed and it filled the gaps.
A two-line excerpt quoted, the table below it missed
Fetch returns the full page as Markdown
A challenge on a filing portal or news site drops that angle
Zenrows' stealth gets through Cloudflare, DataDome, Akamai and similar
A pricing page or dashboard with no numbers
Adaptive Stealth Mode renders the page when it needs it
Forty sources in series: an hour per question
Concurrent Fetch calls, or one Batch job
Raw HTML from ten pages crowds out the question
Markdown cuts the markup; Extract returns only the fields
An entitled database the agent cannot reach
The browser_* tools log in and submit forms in a real cloud browser
How to build a deep research agent
- Plan the question: the model splits it into sub-questions and the sources each needs.
- Find candidate URLs with your search tool or a fetched Google results page; keep the ones worth reading.
- Read in parallel with Fetch and Markdown output; Adaptive Stealth Mode escalates to rendering and stealth only when a source needs it.
- Check coverage, then write: re-read for sub-questions without a source, then write each claim with its URL and quote.
# pip install requests
from concurrent.futures import ThreadPoolExecutor
from datetime import datetime, timezone
import requests
def read_page(url):
"""Agent tool: one page as Markdown, with what it needs to cite it."""
params = {
"apikey": "YOUR_ZENROWS_API_KEY",
"url": url,
"mode": "auto", # Adaptive Stealth Mode
"response_type": "markdown",
}
r = requests.get("https://api.zenrows.com/v1/", params=params, timeout=180)
return {
"url": url,
"ok": r.ok,
"fetched_at": datetime.now(timezone.utc).isoformat(),
"markdown": r.text if r.ok else "",
}
def read_many(urls, workers=10):
"""Fan out over the planner's reading list."""
with ThreadPoolExecutor(max_workers=workers) as pool:
return list(pool.map(read_page, urls))
sources = read_many(READING_LIST)
evidence = [s for s in sources if s["ok"] and len(s["markdown"]) > 500]
print(f"{len(evidence)} of {len(sources)} sources usable")# Long reading list: submit it as one Batch job and collect Markdown when it finishes
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"type": "regular",
"status": "closed",
"zenrows_params": { "mode": "auto", "response_type": "markdown" },
"tasks": [
{ "url": "SOURCE_URL_1", "external_id": "subq-1-a" },
{ "url": "SOURCE_URL_2", "external_id": "subq-1-b" },
{ "url": "SOURCE_URL_3", "external_id": "subq-2-a" }
]
}'npx -y @zenrows/cli init
zenrows fetch SOURCE_URL --output markdown
# Long reading list: reading-list.jsonl holds one {"url": "..."} per line
zenrows batch create reading-list.jsonl --output markdown --wait
zenrows batch results <job-id> --out sources.jsonl# Connect once
claude mcp add zenrows -e ZENROWS_API_KEY=YOUR_ZENROWS_API_KEY -- npx -y @zenrows/mcp
# Then ask:
Use Zenrows to research how the EU and UK currently regulate
buy-now-pay-later lending. Read the regulators' own pages and at least
two recent news reports as markdown. For every claim, quote the
sentence and give the URL. List anything you could not confirm.Which Zenrows product fits
- Read each source the agent picks FetchMarkdown per request, protected and dynamic sites included
- Read a long reading list at once BatchHundreds of URLs in one job, results downloaded at the end
- Reach sources behind a login or form BrowserThe agent drives a real cloud browser
- Research inside Claude, Cursor or an LLM API MCP Serverscrape, batch_* and browser_* tools at https://mcp.zenrows.com/mcp
- Give a coding agent the right defaults Agent ToolkitSkills mapping each research step to a Zenrows call
What success looks like
| Measure | What good looks like | How to check it |
|---|---|---|
| Source read rate | Nearly every planned URL returns usable text | 200s above a minimum text length, per run |
| Citation support | Each quote appears in its cited page | Match quotes against stored Markdown |
| Coverage | A primary source per sub-question | Map findings to sub-questions |
| Time to report | Done within what users will wait | Wall-clock time per run and round |
| Retry rounds | Most runs: one round, at most one retry | Rounds and failed sources in agent logs |
Deep research agent FAQ
-
An AI agent that does a researcher's legwork on a broad question: it splits it, searches, reads many sources, fills gaps, and writes a report citing each claim. The model plans and writes; a web data API like Zenrows does the reading.
-
A tool-calling model, a way to find URLs, a way to read them. Give the model a
read_pagetool calling Zenrows Fetch with Markdown output, or connect the Zenrows MCP server forscrapeand the other tools with no code. Prompt it to plan, read in parallel, check coverage and cite. -
Any model with tool calling. The Anthropic and OpenAI APIs call the remote MCP server at
https://mcp.zenrows.com/mcpdirectly; LangChain, LlamaIndex, CrewAI, Mastra and the OpenAI Agents SDK register Fetch as a function tool. The Zenrows docs have a step-by-step guide to a citing research agent. -
Usually from a search tool. It can also fetch a Google results page through Zenrows, or follow a site's listings, like a regulator's publications index. Reading works the same.
-
Keep URL and fetch time on every page, and have the agent quote the sentence behind each claim. Check each quote against the stored Markdown; unmatched claims get another round or leave the report.
-
Yes, for accounts you are entitled to use: the MCP server's
browser_*tools log in, run a search form and read results in one cloud browser session. Without a login, Fetch is faster. -
A primary source per sub-question and two for key claims: often twenty to fifty pages over a few rounds for a market or policy question. Read more only where sources disagree or a sub-question is empty. Fewer, better sources in full beat snippets from a hundred.