How to scrape protected sites in Cursor with Zenrows MCP

Cursor is one of the most widely adopted AI coding tools, with over one million daily active users. Developers use it for coding, documentation, research, competitive analysis, and agent workflows that depend on live web data.

The trouble is what happens when those workflows hit the real web. Cursor's built-in web access pulls from cached or pre-rendered snapshots. On a JavaScript-rendered page it fetches the shell and misses the data. On a bot-protected page it gets blocked and returns a fragment. On a recently updated page it serves content that may be days old.

The failure is usually silent. Your agent returns something, just not what the page contains. A lot of the web that developers need for serious work sits behind those three barriers.

Zenrows MCP closes that gap. You add a JSON config, restart Cursor, and the agent reaches the live web through Zenrows, the web data infrastructure built for dynamic and protected pages. The Zenrows MCP server overview covers everything it supports.

What Cursor's built-in web access does not reach

Three categories break the cached-snapshot approach:

• JavaScript-rendered pages. Content that only exists after the page's JavaScript runs. Your agent fetches the shell, not the data.
• Sites with bot detection. Pages behind Cloudflare, login walls, or anti-bot systems. Your agent is blocked and returns a cached fragment, with no error surfaced.
• Recently updated pages. Documentation, pricing, or API references changed in the last few hours. The cached copy may be days old with nothing marking it stale.

Adding Zenrows MCP to Cursor

Open Cursor Settings and search for MCP to find the remote MCP configuration section.

You need:

• Node.js v18 or later. Run node --version to confirm.
• A Zenrows API key from app.zenrows.com.
• Cursor installed.

The mcp.json path depends on your operating system:

1. Create the mcp.json file

On macOS and Linux, this creates the config in one command. On Windows, open %USERPROFILE%\.cursor\ in File Explorer, create mcp.json, and paste the same JSON.

mkdir -p ~/.cursor && cat > ~/.cursor/mcp.json << 'EOF'
{
  "mcpServers": {
    "zenrows": {
      "command": "npx",
      "args": ["-y", "@zenrows/mcp"],
      "env": {
        "ZENROWS_API_KEY": "your_api_key_here"
      }
    }
  }
}
EOF

2. Validate the JSON

node -e "JSON.parse(require('fs').readFileSync(require('os').homedir()+'/.cursor/mcp.json','utf8')); console.log('JSON valid')"

Run this before opening Cursor so syntax errors surface early. A trailing comma or a missing bracket stops Cursor loading the config, silently. If the file is valid you get JSON valid.

3. Confirm the connection in Cursor

Open Cursor, click the gear icon at the bottom left, search MCP in settings, then go to Tools & MCPs → Home MCP Servers. Confirm zenrows shows a green status dot with its tools and prompts enabled.

Workflow 1: Fetch live data from a protected site

The goal is clean Markdown from a page Cursor's built-in browsing cannot reach, because an anti-bot challenge sits in front of it.

Run this in Cursor's agent:

Use the Zenrows MCP scrape tool to fetch https://www.scrapingcourse.com/antibot-challenge
and return the content as clean Markdown.

Do not use your built-in web browsing. Only use the Zenrows scrape tool.

The agent fetches the scrape tool schema, then returns the page behind the challenge:

The challenge cleared and the page came back as Markdown. Cursor's built-in browsing returns the challenge interstitial for this URL, not the page behind it. That difference is the whole reason to route the request through Zenrows.

Workflow 2: Extract structured data from a JavaScript-rendered page

Same setup, now on a page that ships its listing data as an embedded JSON payload. Zenrows returns the full rendered page, so your agent has the payload to work from and can pass the result into downstream code.

Use the Zenrows MCP scrape tool to fetch https://www.scrapingcourse.com/javascript-rendering
and extract a structured JSON object containing the first 5 products with their name and price.
Do not use your built-in web browsing.

The products load dynamically, so a plain HTTP request returns an empty grid. Zenrows renders the page and returns the populated HTML, and the agent parses the products out of the #product-grid container:

Those five are the first items in the JavaScript-populated grid, and they are only there because the page was rendered before extraction. That JSON is now in a shape the agent can pass straight into a code generation task, which is what workflow 3 does.

Workflow 3: Generate TypeScript from live protected data

The first two workflows fetched pages. This one uses what comes back as input to a code generation task, against a live retail site where the data is split across two page levels.

The category listing carries product names and prices. SKU and model number only exist on each product's own page. So the agent has to fetch the listing, then fetch each product page, then combine both levels. That is the shape of a real scraping job, and the reason it needs reliable access at every step rather than just the first.

Use the Zenrows MCP scrape tool to fetch
https://www.homedepot.com/b/Tools-Woodworking-Tools/N-5yc1vZc2gv?catStyle=ShowProducts

and return the first 5 products as structured JSON with name and price. Then fetch each
product's individual page and add its SKU and model number.
Set proxy country to US. Do not use your built-in web browsing.

Then write a TypeScript function that fetches and displays product details for a given
product name.

proxy_country: US routes the request through a US IP, which is what makes a US retailer return its normal catalogue regardless of where you run Cursor.

The agent fetches the listing, then fetches each of the five product pages for the fields that only exist there:

{
  "source_url": "https://www.homedepot.com/b/Tools-Woodworking-Tools/N-5yc1vZc2gv?catStyle=ShowProducts",
  "products": [
    {
      "name": "Gorilla 4 fl. oz. Wood Glue",
      "price": "$3.98",
      "sku": "1003827526",
      "model_number": "62020"
    },
    {
      "name": "DEWALT 20V MAX XR Cordless Brushless Fixed Base Compact Router (Tool Only)",
      "price": "$249.00",
      "sku": "1004095707",
      "model_number": "DCW600B"
    },
    {
      "name": "Titebond 8 oz. Original Wood Glue",
      "price": "$3.69",
      "sku": "676828",
      "model_number": "5063"
    },
    {
      "name": "RYOBI Shank Carbide Router Bit Set (15-Piece)",
      "price": "$69.97",
      "sku": "492240",
      "model_number": "A25R151"
    },
    {
      "name": "Milwaukee M18 FUEL 18V Lithium-Ion Brushless Cordless Compact Router (Tool-Only)",
      "price": "$209.00",
      "sku": "1004522892",
      "model_number": "2723-20"
    }
  ]
}

On Home Depot product pages, Store SKU # maps to sku and Model # maps to model_number. Neither appears on the category listing, so every one of those values came from a separate fetch. Each of those passes returned real content, which is what lets the agent map the structure and write code that handles the multi-page case.

It then generated the function:

export async function fetchAndDisplayProductDetails(
  productName: string,
  apiKey = process.env.ZENROWS_API_KEY ?? ""
): Promise<HomeDepotProductDetails | null>

It scrapes the category listing, finds a product by partial name match, fetches that product's page for SKU and model number, and prints the result. Run it with:

export ZENROWS_API_KEY="your_api_key_here"
npm run build
npm run homedepot -- "Gorilla 4 fl. oz. Wood Glue"

Scraped data is saved to data/homedepot-products.json.

Should you use Zenrows or Firecrawl in Cursor?

Both work inside Cursor, and Firecrawl is a light way to turn common pages into clean Markdown.

Zenrows is the better fit when success rates, protected access, high-volume jobs, advanced extraction, and production reliability matter more than simplicity, which is most of what an agent workflow runs into once it leaves the open web. Zenrows runs at a 99.93% success rate across supported targets, and that is the difference that shows up on the protected and heavily rendered pages below.

For the full breakdown across pricing, features, and use cases, read the Zenrows vs Firecrawl comparison.

What you have now

Cursor has Zenrows MCP connected and verified across three workflows you can use directly. On the antibot challenge page, the scrape tool cleared the challenge and returned the page behind it as Markdown. On the JavaScript rendering page, Zenrows rendered the page so the grid was populated and the agent parsed five products out of it. On Home Depot, it gave the agent reliable access across a category listing and five product pages, so it could combine both levels and generate a working TypeScript function from live data.

That closes most of the gap in Cursor's built-in web access. Protected sites, JavaScript-rendered pages, and freshly updated content are all reachable from the same agent workflow.

Cursor is one of several editors and agents this works in. The Zenrows MCP setup guide covers Claude Desktop and other agents, and the Cursor integration docs carry the reference configuration. If you would rather register a tool in code than run an MCP server, the smolagents and OpenAI Agents SDK guides do it that way.

FAQ and debugging

sudo chown -R $(id -u):$(id -g) ~/.npm