---
title: "Competitor price scraping"
description: "Price scraping for competitor price monitoring: selling price, promotions, shipping and stock from Amazon, Walmart, Target and grocery sites, as JSON."
canonical: https://www.zenrows.com/use-cases/price-scraping
updated: 2026-10-18
---

# Competitor price scraping

Price scraping is the automated collection of competitor prices from online stores and marketplaces, including the selling price, list price, promotion, shipping cost and stock for each product you track. Zenrows fetches those product pages from protected retail sites, returns the price fields as JSON, and refreshes your full SKU list on a schedule with Batch.

## Sample output: one competitor price point

```json
{
  "your_sku": "AUR-JKT-M-BLK",
  "competitor": "example-outdoor.com",
  "url": "https://www.example-outdoor.com/p/aurora-rider-jacket",
  "price": 119.0,
  "list_price": 149.0,
  "currency": "USD",
  "promotion": "20% off with code FALL20",
  "shipping": 0.0,
  "delivered_price": 119.0,
  "in_stock": true,
  "country": "us",
  "captured_at": "2026-10-17T06:00:00Z",
  "previous": { "price": 129.0, "captured_at": "2026-10-16T06:00:00Z" },
  "change": -10.0
}
```

An illustrative price point after your diff step: the fields come from CSS selectors you name, and the `previous` and `change` values come from comparing with the last run.

## What price monitoring teams collect

Price monitoring starts with a list of your products and the competitor URLs that sell the same item. Everything else is how often you read those pages and which fields you keep.

| Job | Data you collect | Typical sources |
|---|---|---|
| Competitor price tracking | Selling price and list price per matched SKU | Amazon, Walmart, Target, Best Buy, Home Depot, competitor stores |
| Promotion and markdown tracking | Sale badges, coupon text, bundle offers, start and end dates | Product pages and deal pages such as Amazon Today's Deals |
| Delivered price | Shipping fee, free-shipping threshold, delivery estimate | Product and cart pages of each competitor |
| Marketplace offers | Lowest offer, number of sellers, featured offer seller | Amazon offer listings, Walmart Marketplace, eBay, Mercado Livre |
| Regional and country pricing | Local price and currency per market | MediaMarkt, Carrefour, Allegro, Flipkart, country domains of Amazon |
| Grocery price tracking | Shelf price, unit price, promotion, by store or postcode | Tesco, Sainsbury's, Carrefour, Woolworths, Instacart |
| Price history | Every captured price with a timestamp | Your own store of past runs |

## Why price scraping breaks

Prices are the field retailers protect hardest and change most often. Most failures are quiet: the request succeeds, but the price is missing, stale or from the wrong market.

| What breaks | What it looks like | How Zenrows handles it |
|---|---|---|
| Bot protection on large retailers | A 200 response that holds a CAPTCHA page, or 403s after a few hundred requests | Zenrows' stealth gets through protections that large retailers commonly use, such as AWS WAF, PerimeterX (HUMAN) and Akamai |
| Price rendered after load | The price element is empty or shows a placeholder in the raw HTML | Adaptive Stealth Mode escalates to browser rendering when the price needs it, and returns the page once it is there |
| Price served by an internal API | The page calls a JSON endpoint for price and stock after load | Zenrows captures the page's background network calls (XHR), so you read the price from the response itself |
| Wrong market | You get the US price when you track the UK store, or a different currency | Geotargeting sends the request from the country you are pricing |
| Store or postcode pricing | The competitor's grocery or DIY price differs between the store you priced and the one your customer shops at | Fix the store per task, by cookie or through a Browser session, so each price is tied to one location |
| Sale price versus list price | One selector picks up the struck-through price instead of the selling price | Naming each field with its own CSS selector keeps `price` and `list_price` apart |

## How to scrape competitor prices with Zenrows

1. **Match your SKUs to competitor URLs.** Build a table of your SKU, the competitor, and the product URL on their site. Use GTIN or model number where the page shows one.
2. **Define the price fields per retailer.** Name one set of CSS selectors per competitor for `price`, `list_price`, `promotion`, `shipping` and `stock`.
3. **Test the selectors on a sample.** Send 20 URLs per retailer with Adaptive Stealth Mode, which starts with the lightest setup and escalates to browser rendering and stealth only when a page needs it, and check that every field comes back filled.
4. **Run the full list on a schedule.** Submit all URLs to Batch daily, or several times a day for fast-moving categories, and store every result with a timestamp so you can diff runs.

```python Python
# pip install requests
import json
import requests

# Example selectors: write one set per competitor site
fields = {
    "price": ".product-price .sale",
    "list_price": ".product-price .was",
    "promotion": ".promo-badge",
    "shipping": ".shipping-cost",
    "stock": ".availability",
}
params = {
    "apikey": "YOUR_ZENROWS_API_KEY",
    "url": "COMPETITOR_PRODUCT_URL",
    "mode": "auto",
    "proxy_country": "gb",
    "css_extractor": json.dumps(fields),
}
response = requests.get("https://api.zenrows.com/v1/", params=params)
print(response.json())  # {"price": "£119.00", "list_price": "£149.00", ...}
```

```bash cURL
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
  -H "X-API-Key: YOUR_ZENROWS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "regular",
    "status": "closed",
    "zenrows_params": {
      "mode": "auto",
      "css_extractor": "{\"price\":\".product-price .sale\",\"stock\":\".availability\"}"
    },
    "tasks": [
      { "url": "COMPETITOR_URL_1", "external_id": "AUR-JKT-M-BLK" },
      { "url": "COMPETITOR_URL_2", "external_id": "AUR-JKT-L-BLK" }
    ]
  }'
```

```bash CLI
npx -y @zenrows/cli init
zenrows extract COMPETITOR_PRODUCT_URL --proxy-country gb \
  --css '{"price":".product-price .sale","list_price":".product-price .was","stock":".availability"}'
# skus.jsonl: one {"url": "...", "external_id": "YOUR_SKU"} per line
zenrows batch create skus.jsonl --wait
zenrows batch results <job-id> --out prices-today.jsonl
```

```text Claude Code (MCP)
# Connect once
claude mcp add zenrows -e ZENROWS_API_KEY=YOUR_ZENROWS_API_KEY -- npx -y @zenrows/mcp

# Then ask:
Use Zenrows to check the price, list price and stock of this jacket on John Lewis,
Next and ASOS (URLs below), and give me a table sorted by price, with the
difference against our price of £129.
```

## Which Zenrows product fits

| If you need to | Use | Why |
|---|---|---|
| Check one competitor price on demand | [Fetch](/products/fetch) | One request returns the rendered page or the named price fields |
| Read prices from many store layouts without selectors | [Extract](/products/extract) | Returns price, availability and product fields as JSON |
| Refresh thousands of SKUs every day | [Batch](/products/batch) | The full URL list runs as one job; `external_id` carries your SKU through to the results |
| Get a price that only appears after a postcode or a size choice | [Browser](/products/browser) | Playwright or Puppeteer complete the steps in a real browser |
| Let an analyst ask for a quick price check | [MCP Server](/mcp) | Claude and other MCP clients call Zenrows and return a table |

## What success looks like

| Measure | What good looks like | How to check it |
|---|---|---|
| Match rate | Every tracked SKU has a matched URL at each competitor that sells it | Share of SKUs with a live competitor URL, per competitor |
| Price accuracy | The captured price equals the price a shopper sees in that market | Weekly spot-check of a random sample in a browser from the same country |
| Freshness | Every price is younger than your refresh interval | Alert on any SKU whose latest `captured_at` is older than one cycle |
| Field completeness | Price is present on nearly every successful record | Share of records with an empty `price`, per retailer and per run |
| Change detection | Price drops and new promotions reach the pricing team within one cycle | Compare alert timestamps with the capture timestamps of the change |
| Run duration | The daily refresh finishes well before the pricing team starts work | Log start and end time of each Batch job |

## Price scraping and monitoring FAQ

### What is price scraping?

Price scraping is the automated collection of prices from websites. For retail teams it means reading competitor product pages on a schedule and storing the selling price, list price, promotion, shipping cost and stock for each product they track. The result is a price history per product and competitor that feeds pricing decisions and reports.

### Should I use a price monitoring tool or a price scraping API?

A price monitoring tool gives you dashboards and matching for a monthly fee, with the retailers and refresh rates it supports. A price scraping API gives you the raw price data for any retailer you choose, at the cadence you choose, inside your own warehouse. Teams with a data engineer and specific retailers usually build on an API; teams that want a screen to look at usually buy a tool.

### How often should I scrape competitor prices?

Match the cadence to how often prices move. Daily works for most categories. Electronics, marketplaces and grocery promotions often move several times a day, so those teams refresh every few hours. Before peak events such as Black Friday, increase the frequency for your top sellers only.

### How do I match my products to competitor product pages?

Use a shared identifier first: GTIN, UPC, EAN or the manufacturer part number, which many product pages display. Where there is none, match on brand plus model and check the result by hand for your top sellers. The product matching page covers the full process.

### Can I scrape prices from Amazon and Walmart?

Yes. Send both through Adaptive Stealth Mode, which escalates to browser rendering and Zenrows' stealth when the page needs it, and use geotargeting to request each marketplace from its own country. For marketplace offers, read the offer listing page as well as the product page, since the featured price can come from a different seller each day.

### What is price intelligence?

Price intelligence is the analysis built on top of scraped prices: your price index against each competitor, the share of SKUs where you are cheapest, promotion frequency and depth, and how fast competitors follow your changes. Scraping supplies the raw observations. The intelligence comes from matching them to your catalogue, storing every run, and comparing the series over time.

### Can I track grocery prices by store or postcode?

Yes. Pick the competitor stores that face your own, for example the nearest superstore to each of your branches, and price your basket in each one as a separate Batch task with the store ID in external_id . That gives you a price gap per trade area rather than one national number. The retail data page covers how chains set the store and how to pass it with each request.

### Is price scraping legal?

Collecting publicly displayed prices is widespread in retail and is how most price intelligence is built. Review each site's terms, keep request rates reasonable, and collect product data only, not personal data. For your specific use and jurisdiction, ask your legal team.

## Related use cases

- [MAP monitoring](https://www.zenrows.com/use-cases/map-monitoring.md): Advertised prices and sellers across marketplaces and resellers, with screenshot evidence of every MAP violation.
- [Dynamic pricing data](https://www.zenrows.com/use-cases/dynamic-pricing.md): Live competitor price, stock and demand inputs for a repricing engine, on demand or on a schedule.
- [Ecommerce web scraping](https://www.zenrows.com/use-cases/ecommerce-web-scraping.md): Product, price, stock, review and seller data from Amazon, Walmart, eBay and Shopify stores, as JSON.
- [Competitor monitoring](https://www.zenrows.com/use-cases/competitor-monitoring.md): Pricing pages, changelogs, feature pages and job posts on rival sites, snapshotted and diffed on every run.

Start free: https://app.zenrows.com/register. Docs: https://docs.zenrows.com