How to Build Your Own Instant Data Scraper

Do you still treat web scraping as an advanced coding task? You can avoid heavy coding by building your own instant data scraper using low-code automation tools like n8n, Zapier, Make, etc, along with a web scraping API.

Wiring the nodes is not the hard part, but keeping runs consistent across JavaScript pages, challenge pages, and targets that change when IPs or sessions change is challenging. In this tutorial, you’ll learn how to build a resilient instant data scraper with Google Sheets, n8n, and ZenRows.

How Instant Data Scraping Works

An instant data scraper follows a simple pattern. An input event, like a new URL being added to your input source, triggers an automated run. The workflow fetches that URL, scrapes the target site, then writes the output to your destination as structured data.

Most instant data scrapers have a common point of failure. The scraping node is usually blocked by anti-bots, or it returns incomplete data when the target renders content dynamically with JavaScript. This makes the setup feel inconsistent across targets.

ZenRows provides a Universal Scraper API with a single endpoint that includes all the capabilities needed for reliable scraping in low-code workflows. Its Adaptive Stealth Mode, enabled with mode=auto, optimizes your scraping request for success at the lowest possible cost, based on the target's requirements.

Auto mode decides when to use JavaScript rendering and Premium Proxies, so you don’t have to guess per target. If you prefer a custom configuration, you can select specific features, such as JavaScript rendering for heavy client-side pages, Premium Proxies for anti-bot-protected targets, session management to keep a consistent IP address across related requests, etc.

Building The Instant Data Scraper

In this section, you’ll build an instant data scraper where adding a new URL row in Google Sheets triggers a run in n8n. It then uses ZenRows to scrape the target site and stores the extracted results back in a Google Sheets file.

Step 1: Build The Google Sheets Setup

The first step in building the instant data scraper is to set up the Google Sheet that your workflow will read data from and write results to. You'll need two tabs. A "Queue" tab to store URLs waiting to be scraped and their run status, and a "Results" tab to store the extracted output. To achieve this:

• Create a Google Sheets file named "Instant Scraper Workflow".
• Then, rename the default tab to "Queue" and create a second tab named "Results".

Next, add the columns the workflow needs to track each URL and its run state.

• In the "Queue" tab, create these columns in the first row: row_id, url, css_selectors, status, error_message.

row_id is the identifier the workflow uses to update the correct URL entry. url is the target page you want to scrape. css_selectors is where you define what to extract from that page by mapping field names to CSS selectors. status tracks the run state, and error_message stores a short failure reason when a run fails.

Step 2: Build the CSS Selector Payload

Next, build the CSS selector payload for each target inside the ZenRows Request Builder. This tells the instant scraper what fields to extract when it processes a URL from the queue.

In this guide, we'll use the ScrapingCourse e-commerce category page and the Antibot Challenge page as our targets. We recommend testing your selectors in the ZenRows Request Builder before you paste them into Google Sheets. This helps you confirm the selectors work and enables you to avoid workflow failures later.

Follow the steps below to build out your selectors using ZenRows' css_extractor parameter:

1. Sign up for ZenRows.
2. Then open the ZenRows Request Builder.
3. Paste the ScrapingCourse e-commerce URL into the URL field.

4. Enable "Adaptive Stealth Mode".
5. Under the "Output" section, click "More", then select "Parsers".
6. Paste the css_extractor JSON below in this field.

{"title":"h2.woocommerce-loop-product__title","price":".price .woocommerce-Price-amount","image":"img @src"}

7. Click "Run & Preview Results".
8. Confirm the output returns arrays for title, price, and image.

{
  "title": [
    "Abominable Hoodie",
    "Adrienne Trek Jacket"
    <!--ommited for brevity-->
  ],
  "price": [
    "$69.00",
    "$57.00"
    <!--ommited for brevity-->
  ],
  "image": [
    "https://www.scrapingcourse.com/ecommerce/wp-content/uploads/2024/03/mh09-blue_main.jpg",
    "https://www.scrapingcourse.com/ecommerce/wp-content/uploads/2024/03/wj08-gray_main.jpg"
    <!--ommited for brevity-->
  ]
}

This confirms the selectors you used are extracting the right data.

Repeat the same Request Builder test for the Antibot Challenge page using the selectors below.

{"title":"h1.page-title","message":".challenge-title"}

You should see results similar to this, confirming the selectors are working.

{
  "message": "You bypassed the Antibot challenge! :D",
  "title": "Antibot Challenge"
}

Now, add both targets to the Queue. These entries will be picked up later when you run the n8n workflow:

1. Go back to the "Queue" tab.
2. Add two new rows.
3. Set row_id to a unique value for each row.
4. Paste the target URLs into the url field.
5. Paste the corresponding css_extractor JSON string into the css_selectors field.
6. Set status to pending for each entry.
7. Leave error_message empty for each entry.

Your input schema is ready! Now, you need to set up the output schema for the workflow to write scraping results to.

In the "Results" tab, create these columns in the first row: run_id, source_url, results.

run_id links each extracted record back to the row_id in Queue. source_url stores the scraped URL, and results stores the extracted payload returned by ZenRows.

You do not need to fill in any data in the "Results" tab. The workflow automatically fills run_id, source_url, and results.

Your Google Sheet is ready. Let's now connect it to n8n.

Step 3: Connect Google Sheets to n8n

To connect n8n to the Google Sheet you created, give n8n permission to read and update rows in the "Instant Scraper Workflow" spreadsheet.

1. Log in to your n8n account. In the left sidebar, click Personal, then open Credentials.
2. Click "Create credential". This opens a credentials selection window.

3. Search "sheet". Then, select the "Google Sheets Trigger OAuth2 API" credential.

4. Sign in with the Google account that has access to your "Instant Scraper Workflow" spreadsheet, approve the consent prompt, then save the credentials.

5. Create a second credential for "Google Sheets OAuth2 API". Sign in with the same Google account, approve the consent prompt, then save it. When the credential shows a green status, the connection succeeded.

Your Google Sheets account and n8n are now connected. Let's start building the Instant Data Scraper workflow.

Step 4: Create The Instant Data Scraper Workflow Canvas

Create a new workflow canvas where you’ll add the nodes for the scraper.

1. Under Personal, click Workflows.
2. Then click "Create Workflow" in the top right. n8n opens a blank canvas in a new view.

3. In the top left, rename the workflow to "Instant Data Scraper".

The workflow creation confirmation message at the bottom-right corner indicates that the canvas was created successfully.

Step 5: Set Up The Triggers

An n8n workflow starts with a trigger. It can run on a schedule, via a webhook, or when an event happens.

In this workflow, you’ll use a Google Sheets trigger to start a run when a new row is added to the "Queue" tab, and a manual trigger to run tests on demand.

1. In the workflow canvas, click the "Add first step…" plus icon to add your first trigger.

2. Then search for "Google Sheets Trigger" and select. Choose "On row added".

3. On the node configuration that opens, rename the node to "Watch for New Pending URLs".
4. Select the "Google Sheets Trigger OAuth2 API" credential that you've set up previously.
5. Then set Document to "Instant Scraper Workflow", Sheet to "Queue", and "Trigger On" to "Row Added". This trigger starts a run whenever a new entry is added to the "Queue" tab in the "Instant Scraper Workflow" sheet.

6. Proceed to the canvas and click the "+" icon in the top right corner.

7. In the search bar that appears, search for "Manual Trigger".

8. Rename it to "Start Workflow". This trigger lets you run the workflow on demand during testing.

Your workflow should now have two triggers: "Watch for New Pending URLs" and "Start Workflow".

Step 6: Read Pending URLs From The Queue

The Google Sheets Trigger only returns the newly added entry. This starts the workflow execution, but it doesn’t guarantee you’ll process every URL already waiting in "pending status". If multiple URLs were added between polls, or a previous run stopped early, those items can sit in the queue until you explicitly read them. To avoid this:

1. In the canvas, click "+" next to your Google Sheets trigger node.
2. Search for Google Sheets in the node search box.
3. Then select the Google Sheets' "Get rows(s) in sheet" action.

4. Rename the Google Sheets node to "Read Pending URLs from Queue".
5. Again, connect to your "Google Sheets account".
6. Under Document, select the "Instant Scraper Workflow" spreadsheet and set the Sheet dropdown to the Queue Sheet.
7. Set the Filters to where status equals pending so each execution retrieves all URLs still pending for scraping.

8. Then connect the manual trigger to this node.

Your workflow should be similar to the one below at this point.

Step 7: Update the URL Status to Processing

You need to make the execution state visible before scraping begins. This prevents the same URL from being picked up again while it is being executed.

1. Click the "+" icon next to "Read Pending URLs from Queue".
2. Search for "Google Sheets", and select the "Update row in sheet" action.

3. Rename the node to "Mark as Processing".
4. Select your "Google Sheets OAuth2 API" credential again, then set "Document" to Instant Scraper Workflow and "Sheet" to "Queue".
5. In "Columns", set status to processing and set row_id to {{ $json.row_id }}.
6. In "Matching Columns", select row_id so n8n updates the correct entry.

{{ $json.row_id }} reads the row_id value from the current item, so the update applies to the same URL this run is handling.

Step 8: Scrape Using ZenRows

To scrape each queued URL with ZenRows, you first need to integrate ZenRows with n8n.

Featured

How to Integrate Web Scraping Into Your n8n Workflows

Learn to enhance your n8n workflow with real-time web data. Optimize your pipeline for consistent web access, scalability, and high-quality results.

• Begin by opening the ZenRows Request Builder.
• Leave the URL field empty.
• Enable "Adaptive Stealth Mode".
• Choose the API connection mode and select cURL as your programming language.

• Copy the generated cURL code and head back to n8n.

Here is an example of the generated code:

curl "https://api.zenrows.com/v1/?apikey=<YOUR_ZENROWS_API_KEY>&url=&mode=auto"

In your n8n canvas:

1. Click the "+" icon after "Mark as Processing".
2. Search for the "HTTP Request" node and select it.

3. Rename the node to "Scrape Using ZenRows".
4. Click "Import cURL".
5. Paste the generated ZenRows cURL code in the cURL Command field.

6. Click "Import". This auto-populates the HTTP Request node fields with your ZenRows query parameters, including the apikey, url, and mode.

Now update the HTTP Request node to use the dynamic query parameters supplied by your workflow.

1. In Query Parameters, set url to {{ $('Read Pending URLs from Queue').item.json.url }}. This pulls the url value from the current row returned by "Read Pending URLs from Queue".
2. Click "Add parameter" and add css_extractor.
3. Set css_extractor to {{ $('Read Pending URLs from Queue').item.json.css_selectors }}.

4. Open the node "Options" and set "Timeout" to 180000 so the request has enough time when the target is slow or JavaScript-heavy.

5. Finally, open the node "Settings" and enable "Continue (using error output)" so the workflow can still capture an error if it occurs and update the execution state in "Queue".

By now, your workflow should look like the one below, with the "Scrape Using ZenRows" node displaying both the "Success" and "Error" output modes.

Step 9: Store the Output and Update the Run Status

After receiving the scraping results from ZenRows, you need to do three things.

1. First, store the extracted payload in the "Results" sheet.
2. Update the URL state in "Queue" to completed when the scrape succeeds.
3. If the scrape fails, update the same URL entry to failed and write the failure reason to error_message.

Let's start by storing the extracted data in the "Results" sheet.

1. Click the "+" icon on the success output of "Scrape Using ZenRows".
2. Search for "Google Sheets" in the node search bar and select the "Append row in sheet" action.

3. Rename the node to "Append to Results".
4. Select your "Google Sheets OAuth2 API" credential.
5. Then set "Document" to "Instant Scraper Workflow" and "Sheet" to "Results".
6. In "Columns", set run_id to {{ $('Read Pending URLs from Queue').item.json.row_id }} so each result links back to the same Queue entry.
7. Set source_url to {{ $('Read Pending URLs from Queue').item.json.url }} so you store the original URL.
8. Set results to {{ $json.data }} so you write the extracted JSON returned by ZenRows.

After writing the results, you need to mark the scraping as completed in the "queue" tab:

1. Click the "+" icon next to "Append to Results", search for "Google Sheets", and select the "Update row in sheet" action.

2. Then, rename the node to "Mark as Completed".
3. Set "Document" to "Instant Scraper Workflow" and "Sheet" to "Queue".
4. In "Columns", set status to completed.
5. Set row_id to {{ $('Read Pending URLs from Queue').item.json.row_id }}. This updates the same Queue entry that produced the result.
6. Finally, set error_message to "NA".

But what happens if the scraping flow returns an error, such as when you submit incorrect CSS selectors? In such a situation, route the error output from "Scrape Using ZenRows" to a Google Sheets update node. To achieve this:

1. Click the "+" icon on the error output of "Scrape Using ZenRows".
2. Search for "Google Sheets", and select the "Update row in sheet" action.

3. Rename the node to "Log Error to Queue".
4. Then, set "Document" to "Instant Scraper Workflow" and "Sheet" to "Queue".
5. In "Columns", set row_id to {{ $('Read Pending URLs from Queue').item.json.row_id }}.
6. Set status to failed.
7. Set error_message to {{ $json.error.message }}. This reads the error message from the failed HTTP node output and stores it against the same Queue entry.
8. In "Matching Columns", select row_id.

9. Save the workflow by clicking the "Save " button in the top-right corner of your canvas window.

You have now completed creating the instant data scraper, and the full workflow should look like this.

Bravo! You now have an instant data scraper that’s ready to scrape any target site.

Run Your Instant Data Scraper

Make the workflow live so the Google Sheets trigger can fire. To achieve this:

1. Click "Publish" in the top bar.

2. Then, in the pop-up that appears, provide a description of your workflow.
3. Click the second "Publish" button. Published workflows run automatically whenever their trigger conditions are met.

Next, test the instant data scraper using the Scraping Course anti-bot challenge page for protected sites and the e-commerce page.

Click the "Execute workflow" button, which is next to the "Start Workflow" trigger.

On a successful run, you’ll see the ZenRows scraped data appended to "Results", and the status column in the "Queue" tab updated to completed.

And the Results sheet looks like this. Feel free to split the scraped data:

Congratulations! 🎉 Your ZenRows-powered instant data scraper just scraped an anti-bot-protected website.

If a run fails, you'll see the status updated to failed and an error_message, logged so you can fix the URL or selectors.

Troubleshooting Your Instant Data Scraper

If the workflow doesn’t start, or a row stays stuck in pending, the problem is usually in the trigger, the queue filter, or the update step. Use the table below to identify the issue and apply the fix.

Conclusion

In this article, you’ve learned how to build an instant data scraper that queues URLs in Google Sheets, uses n8n as the workflow runner, and calls ZenRows for scraping and extraction. You also tested it against both a normal page and an anti-bot challenge target.

ZenRows makes this setup reliable by handling JavaScript pages and anti-bot protection behind a single endpoint. When you enable Adaptive Stealth Mode, ZenRows automatically selects the lowest-cost settings that maintain the highest success rate for your target. You only pay for what’s used.

Try ZenRows for free now or speak with sales!