# How to scrape protected sites in Cursor with Zenrows MCP

> Cursor's built-in web access reads cached snapshots, so it misses JS-rendered data and gets blocked on protected pages. Add Zenrows MCP and run three workflows on live sites.

Source: https://www.zenrows.com/blog/scrape-protected-sites-cursor

Cursor is one of the most widely adopted AI coding tools, with [over one million daily active users](https://www.getpanto.ai/blog/cursor-ai-statistics). Developers use it for coding, documentation, research, competitive analysis, and agent workflows that depend on live web data.

The trouble is what happens when those workflows hit the real web. Cursor's built-in web access pulls from cached or pre-rendered snapshots. On a JavaScript-rendered page it fetches the shell and misses the data. On a bot-protected page it gets blocked and returns a fragment. On a recently updated page it serves content that may be days old.

The failure is usually silent. Your agent returns something, just not what the page contains. A lot of the web that developers need for serious work sits behind those three barriers.

Zenrows MCP closes that gap. You add a JSON config, restart Cursor, and the agent reaches the live web through Zenrows, the web data infrastructure built for dynamic and protected pages. The [Zenrows MCP server overview](https://docs.zenrows.com/mcp/overview) covers everything it supports.

## What Cursor's built-in web access does not reach

Three categories break the cached-snapshot approach:

- **JavaScript-rendered pages.** Content that only exists after the page's JavaScript runs. Your agent fetches the shell, not the data.
- **Sites with bot detection.** Pages behind Cloudflare, login walls, or anti-bot systems. Your agent is blocked and returns a cached fragment, with no error surfaced.
- **Recently updated pages.** Documentation, pricing, or API references changed in the last few hours. The cached copy may be days old with nothing marking it stale.

## Adding Zenrows MCP to Cursor

Open Cursor Settings and search for MCP to find the remote MCP configuration section.

![Cursor Settings with "mcp" typed in the search field, showing the Tools and MCPs results with Home MCP Servers highlighted](/blog/_img/cursor-settings-mcp-search.png)

You need:

- Node.js v18 or later. Run `node --version` to confirm.
- A Zenrows API key from [app.zenrows.com](https://app.zenrows.com/register).
- Cursor installed.

The `mcp.json` path depends on your operating system:

| Operating system | MCP configuration path |
| --- | --- |
| macOS | `~/.cursor/mcp.json` |
| Windows | `%APPDATA%\Cursor\mcp.json` |
| Linux | `~/.cursor/mcp.json` |

### 1. Create the mcp.json file

On macOS and Linux, this creates the config in one command. On Windows, open `%USERPROFILE%\.cursor\` in File Explorer, create `mcp.json`, and paste the same JSON.

```bash
mkdir -p ~/.cursor && cat > ~/.cursor/mcp.json << 'EOF'
{
  "mcpServers": {
    "zenrows": {
      "command": "npx",
      "args": ["-y", "@zenrows/mcp"],
      "env": {
        "ZENROWS_API_KEY": "your_api_key_here"
      }
    }
  }
}
EOF
```

### 2. Validate the JSON

```bash
node -e "JSON.parse(require('fs').readFileSync(require('os').homedir()+'/.cursor/mcp.json','utf8')); console.log('JSON valid')"
```

Run this before opening Cursor so syntax errors surface early. A trailing comma or a missing bracket stops Cursor loading the config, silently. If the file is valid you get `JSON valid`.

### 3. Confirm the connection in Cursor

Open Cursor, click the gear icon at the bottom left, search MCP in settings, then go to **Tools & MCPs → Home MCP Servers**. Confirm `zenrows` shows a green status dot with its tools and prompts enabled.

![Cursor's Home MCP Servers panel listing the zenrows server with a green status dot and "37 tools, 3 prompts enabled"](/blog/_img/cursor-zenrows-mcp-connected.png)

## Workflow 1: Fetch live data from a protected site

The goal is clean Markdown from a page Cursor's built-in browsing cannot reach, because an anti-bot challenge sits in front of it.

Run this in Cursor's agent:

```text
Use the Zenrows MCP scrape tool to fetch https://www.scrapingcourse.com/antibot-challenge
and return the content as clean Markdown.

Do not use your built-in web browsing. Only use the Zenrows scrape tool.
```

The agent fetches the scrape tool schema, then returns the page behind the challenge:

![Cursor's agent output showing the page content fetched through Zenrows with js_render and premium_proxy enabled, rendering the Scraping Course "You bypassed the Antibot challenge" page as Markdown](/blog/_img/cursor-zenrows-antibot-challenge-markdown.png)

The challenge cleared and the page came back as Markdown. Cursor's built-in browsing returns the challenge interstitial for this URL, not the page behind it. That difference is the whole reason to route the request through Zenrows.

## Workflow 2: Extract structured data from a JavaScript-rendered page

Same setup, now on a page that ships its listing data as an embedded JSON payload. Zenrows returns the full rendered page, so your agent has the payload to work from and can pass the result into downstream code.

```text
Use the Zenrows MCP scrape tool to fetch https://www.scrapingcourse.com/javascript-rendering
and extract a structured JSON object containing the first 5 products with their name and price.
Do not use your built-in web browsing.
```

The products load dynamically, so a plain HTTP request returns an empty grid. Zenrows renders the page and returns the populated HTML, and the agent parses the products out of the `#product-grid` container:

![Cursor's agent output with a JSON object of five products and prices parsed from the rendered #product-grid container, noting the fetch used js_render and premium_proxy](/blog/_img/cursor-zenrows-javascript-rendered-products-json.png)

Those five are the first items in the JavaScript-populated grid, and they are only there because the page was rendered before extraction. That JSON is now in a shape the agent can pass straight into a code generation task, which is what workflow 3 does.

## Workflow 3: Generate TypeScript from live protected data

The first two workflows fetched pages. This one uses what comes back as input to a code generation task, against a live retail site where the data is split across two page levels.

The category listing carries product names and prices. SKU and model number only exist on each product's own page. So the agent has to fetch the listing, then fetch each product page, then combine both levels. That is the shape of a real scraping job, and the reason it needs reliable access at every step rather than just the first.

```text
Use the Zenrows MCP scrape tool to fetch
https://www.homedepot.com/b/Tools-Woodworking-Tools/N-5yc1vZc2gv?catStyle=ShowProducts

and return the first 5 products as structured JSON with name and price. Then fetch each
product's individual page and add its SKU and model number.
Set proxy country to US. Do not use your built-in web browsing.

Then write a TypeScript function that fetches and displays product details for a given
product name.
```

`proxy_country: US` routes the request through a US IP, which is what makes a US retailer return its normal catalogue regardless of where you run Cursor.

The agent fetches the listing, then fetches each of the five product pages for the fields that only exist there:

```json
{
  "source_url": "https://www.homedepot.com/b/Tools-Woodworking-Tools/N-5yc1vZc2gv?catStyle=ShowProducts",
  "products": [
    {
      "name": "Gorilla 4 fl. oz. Wood Glue",
      "price": "$3.98",
      "sku": "1003827526",
      "model_number": "62020"
    },
    {
      "name": "DEWALT 20V MAX XR Cordless Brushless Fixed Base Compact Router (Tool Only)",
      "price": "$249.00",
      "sku": "1004095707",
      "model_number": "DCW600B"
    },
    {
      "name": "Titebond 8 oz. Original Wood Glue",
      "price": "$3.69",
      "sku": "676828",
      "model_number": "5063"
    },
    {
      "name": "RYOBI Shank Carbide Router Bit Set (15-Piece)",
      "price": "$69.97",
      "sku": "492240",
      "model_number": "A25R151"
    },
    {
      "name": "Milwaukee M18 FUEL 18V Lithium-Ion Brushless Cordless Compact Router (Tool-Only)",
      "price": "$209.00",
      "sku": "1004522892",
      "model_number": "2723-20"
    }
  ]
}
```

On Home Depot product pages, Store SKU # maps to `sku` and Model # maps to `model_number`. Neither appears on the category listing, so every one of those values came from a separate fetch. Each of those passes returned real content, which is what lets the agent map the structure and write code that handles the multi-page case.

It then generated the function:

```ts
export async function fetchAndDisplayProductDetails(
  productName: string,
  apiKey = process.env.ZENROWS_API_KEY ?? ""
): Promise<HomeDepotProductDetails | null>
```

It scrapes the category listing, finds a product by partial name match, fetches that product's page for SKU and model number, and prints the result. Run it with:

```bash
export ZENROWS_API_KEY="your_api_key_here"
npm run build
npm run homedepot -- "Gorilla 4 fl. oz. Wood Glue"
```

Scraped data is saved to `data/homedepot-products.json`.

![Cursor's file review panel showing two files changed, homedepotProductDetails.ts and homedepot-products.json, with their line counts](/blog/_img/cursor-homedepot-typescript-files-changed.png)

## Should you use Zenrows or Firecrawl in Cursor?

Both work inside Cursor, and Firecrawl is a light way to turn common pages into clean Markdown.

Zenrows is the better fit when success rates, protected access, high-volume jobs, advanced extraction, and production reliability matter more than simplicity, which is most of what an agent workflow runs into once it leaves the open web. Zenrows runs at a 99.93% success rate across supported targets, and that is the difference that shows up on the protected and heavily rendered pages below.

| Scenario | Recommended tool | Reason |
| --- | --- | --- |
| Public blog or open documentation site | Firecrawl | Fast, clean Markdown from the open web |
| Multi-page discovery on public sites | Firecrawl | Built-in crawl and map workflows |
| Protected site behind Cloudflare | Zenrows | Premium proxy and JavaScript rendering handle challenge pages |
| JavaScript-rendered page with an embedded data payload | Zenrows | Renders the full page, extracts `__NEXT_DATA__` and similar payloads |
| Recently updated docs or pricing pages | Zenrows | Live fetch from the current page |
| High-volume recurring data workflow | Zenrows | Batch processing, observability, and retry logic built in |
| Agent needs structured JSON output | Zenrows | Returns HTML, Markdown, JSON, screenshots, or plain text |

For the full breakdown across pricing, features, and use cases, read the [Zenrows vs Firecrawl comparison](/blog/best-firecrawl-alternative-for-anti-bot-bypass).

## What you have now

Cursor has Zenrows MCP connected and verified across three workflows you can use directly. On the antibot challenge page, the scrape tool cleared the challenge and returned the page behind it as Markdown. On the JavaScript rendering page, Zenrows rendered the page so the grid was populated and the agent parsed five products out of it. On Home Depot, it gave the agent reliable access across a category listing and five product pages, so it could combine both levels and generate a working TypeScript function from live data.

That closes most of the gap in Cursor's built-in web access. Protected sites, JavaScript-rendered pages, and freshly updated content are all reachable from the same agent workflow.

Cursor is one of several editors and agents this works in. The [Zenrows MCP setup guide](/blog/pull-web-data-into-claude-cursor-and-any-ai-agent-with-zenrows-mcp) covers Claude Desktop and other agents, and the [Cursor integration docs](https://docs.zenrows.com/integrations/cursor) carry the reference configuration. If you would rather register a tool in code than run an MCP server, the [smolagents](/blog/zenrows-smolagents) and [OpenAI Agents SDK](/blog/web-aware-agent-openai-agents-sdk-zenrows) guides do it that way.

## FAQ and debugging

### Where do I find Cursor's MCP settings?

Click the gear icon at the bottom left, search "MCP", then click **Tools & MCPs → Home MCP Servers**. The file lives at `~/.cursor/mcp.json` on macOS and Linux, and `%USERPROFILE%\.cursor\mcp.json` on Windows.

### Zenrows MCP is not appearing after I added the config

Restart Cursor after saving. On macOS use ⌘Q rather than closing the window: the server process only reads your API key when it spawns, so closing the window is not enough. Then validate your JSON with the command from step 2, since a trailing comma or missing bracket stops the config loading without saying so.

### I am getting a 401 error

The server is connected but the key it passes is not valid. You will see `401 AUTH003 Invalid apikey provided` in the agent response. Check that `ZENROWS_API_KEY` in `mcp.json` matches a live key in your Zenrows account and is not still the `your_api_key_here` placeholder. Save, quit Cursor fully, and reopen.

### I am getting an EACCES error on macOS

Restore npm cache ownership, then restart Cursor so it can write to the cache again:

```bash
sudo chown -R $(id -u):$(id -g) ~/.npm
```

### Can I use the hosted Zenrows MCP server instead of npx?

Yes. Use `https://mcp.zenrows.com/mcp` with your API key as a Bearer token in Cursor's remote MCP configuration.