---
title: "Retail data scraping"
description: "Retail data from store locators, store-level prices, weekly flyers and click-and-collect stock at Tesco, Kroger, Walmart, Carrefour and other chains."
canonical: https://www.zenrows.com/use-cases/retail-data
updated: 2026-10-18
---

# Retail data scraping

Retail data scraping collects store-level information from physical and omnichannel retailers: store locations, prices per store, promotions, weekly flyers, assortment and click-and-collect stock. Zenrows fetches retailer sites as a shopper in a chosen store or postcode would see them, through protected access, and returns structured records you can compare across stores, regions and chains.

## Sample output: one product in one store

```json
{
  "chain": "Example Grocer",
  "store": {
    "store_id": "0412",
    "name": "Example Grocer Riverside",
    "postcode": "M4 1AB",
    "format": "superstore"
  },
  "product": {
    "name": "Semi-skimmed milk 4 pints",
    "retailer_code": "254881903",
    "size": "2.272 l"
  },
  "price": 1.65,
  "unit_price": { "value": 0.73, "per": "litre" },
  "loyalty_price": 1.45,
  "promotion": {
    "type": "multibuy",
    "text": "2 for 3.00",
    "valid_from": "2026-10-15",
    "valid_to": "2026-11-04"
  },
  "availability": { "delivery": true, "click_and_collect": true },
  "currency": "GBP",
  "collected_at": "2026-10-17T05:40:00Z"
}
```

An illustrative store-level record: shelf price, unit price, loyalty price and a dated multibuy, all for one store.

## What retail data teams collect

Chains set prices, promotions and stock per store or per region, so every job below starts by choosing which store the website should show.

| Job | Data you collect | Typical sources |
|---|---|---|
| Grocery price index | Shelf, unit and loyalty prices for a fixed basket, per store | Tesco, Sainsbury's, Asda, Kroger, Albertsons, Carrefour, Woolworths |
| Promotion and flyer tracking | Deal text, mechanic, start and end dates, featured products | Weekly ads and offers pages at Aldi, Lidl, Target, Walgreens |
| Store network mapping | Store addresses, formats, opening hours, openings and closures | Chain store locators, Google Maps |
| Regional assortment | Which products each store or region lists | Store-specific category pages |
| Click-and-collect availability | Pickup and delivery options per store | Walmart, Target, Currys, Boots, Leroy Merlin |
| Retail analytics for CPG brands | Prices, promotions and photos for a brand's range across chains | Supermarket, pharmacy and duty-free websites |

## Why store-level retail data is hard to scrape

A retailer's website shows one store at a time. To build a price per store you have to become a shopper in each store, many times, without being blocked.

| What breaks | What it looks like | How Zenrows handles it |
|---|---|---|
| Location set by cookie or session | Every request returns national prices, or a redirect to a store picker | Fetch sends the store cookie as a custom header with each request; Browser sessions click the picker |
| Hundreds of postcodes per chain | A run multiplies by 300 locations and stops finishing on time | Batch submits every product and store combination as one job |
| Bot protection on grocery sites | Queue pages and challenges during promotion weeks | Zenrows' stealth gets through Akamai, Cloudflare and similar protections |
| Datacenter IPs refused | Retailers block cloud IP ranges outright | Adaptive Stealth Mode with geotargeting, so the request comes from the chain's country |
| Flyers built as images or viewers | The weekly ad has no HTML prices to parse | Screenshots keep a dated image of each flyer page |
| Store locators behind maps | Store lists load as the map moves | Crawl collects the per-store pages most chains also publish |

## How to scrape retail data with Zenrows

1. **Build the store list.** Crawl the chain's store directory for per-store pages, or read the locator, and keep store ID, address and postcode.
2. **Learn how the site sets the store.** Check whether the store lives in a cookie, a URL parameter or a session, by changing it once in your browser.
3. **Fetch prices per store.** Send the store cookie as a custom header on each product or category request, or use a Browser session when the picker is interactive.
4. **Capture promotions.** Record deal text and dates from offers pages, and take screenshots of flyers that only exist as images.
5. **Run it as one job.** Submit every store and product pair to Batch on your schedule, daily for promotions and weekly for the full basket.

```bash cURL
curl -X POST "https://api.zenrows.com/v1/crawls" \
  -H "X-API-Key: YOUR_ZENROWS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "STORE_DIRECTORY_URL",
    "max_items": 2000,
    "max_pages": 2000,
    "include_patterns": ["/stores/"],
    "exclude_patterns": ["/stores/search"],
    "output_format": "markdown"
  }'
```

```python Python
# pip install requests
import requests

params = {
    "apikey": "YOUR_ZENROWS_API_KEY",
    "url": "PRODUCT_URL",
    "mode": "auto",  # Adaptive Stealth Mode
    "proxy_country": "gb",
    "custom_headers": "true",
    "extract": "auto",
}
# The store the site should price for, copied from your browser
# (example cookie name: check which one the retailer uses)
headers = {"Cookie": "selected_store=0412"}

response = requests.get("https://api.zenrows.com/v1/", params=params, headers=headers)
print(response.json()["parsed"])
```

```bash CLI
npx -y @zenrows/cli init
# Dated screenshot of this week's flyer page
zenrows fetch FLYER_URL --proxy-country gb --screenshot --out flyer-2026-10-19.png
# Every store and product pair as one job
zenrows batch create store-products.jsonl --wait
zenrows batch results JOB_ID --out prices.jsonl
```

```text Claude Code (MCP)
# Connect once
claude mcp add zenrows -e ZENROWS_API_KEY=YOUR_ZENROWS_API_KEY -- npx -y @zenrows/mcp

# Then ask:
Use Zenrows to open the Tesco website, set the store to the one nearest postcode LS1 4AP, then return the shelf price, Clubcard price, unit price and promotion text for semi-skimmed milk 4 pints, Hovis wholemeal bread 800g and PG Tips 80 tea bags.
```

## Which Zenrows product fits

| If you need to | Use | Why |
|---|---|---|
| Get prices for one store from product pages | [Fetch](/products/fetch) | Store cookies as custom headers, market through geotargeting |
| List every store a chain operates | [Crawl](/products/crawl) | Collects per-store pages from the chain's directory |
| Price a basket across hundreds of stores | [Batch](/products/batch) | Every store and product pair in one job |
| Pick a store through an interactive map or form | [Browser](/products/browser) | Playwright or Puppeteer drives the picker like a shopper |
| Get product fields without writing selectors | [Extract](/products/extract) | Automatic structured extraction returns price, size and promotion as JSON |

## What success looks like

| Measure | What good looks like | How to check it |
|---|---|---|
| Store coverage | Every store in your panel has a price for every basket item | Count missing store and product pairs per run |
| Location accuracy | Prices belong to the store you asked for, not a default | Spot-check a sample against the site with the same store selected |
| Promotion capture | Each active promotion has a mechanic and both dates | Report promotions missing a start or end date |
| Unit price consistency | Prices compare per litre, kilo or count across chains | Validate size parsing and flag records without a unit price |
| Run completion | The daily run finishes before your analysts start | Track Batch job end time against your deadline |
| Failure rate per run | Few store-product pairs need a retry, and the share stays flat as stores are added | Failed tasks per Batch job divided by store-product pairs |

## Retail data FAQ

### What is retail data scraping?

Retail data scraping is collecting public information from retailers' websites about their stores and what they sell in each one: locations, opening hours, prices, promotions, assortment and pickup availability. It covers supermarkets, pharmacies, DIY, electronics and department store chains that sell both in stores and online.

### Where can I get grocery price data?

Most large grocers publish prices online for delivery and click-and-collect, often per store. Collect them from the chain's own website for the stores you need, with the store set in the session or cookie. Keep shelf price, unit price and loyalty price as separate fields, since loyalty pricing is common in UK and US grocery.

### Can I get prices for a specific store?

Yes, when the retailer prices online by store. Set the store the way the website does, usually a cookie or a store picker, and keep it fixed for the request. With Fetch, pass the cookie as a custom header; with Browser, select the store in the page before reading prices.

### How do I collect weekly ads and promotions?

Offers pages usually list deal text and dates as HTML, which Fetch returns directly. Some flyers are images or page-turn viewers; take a screenshot of each page and store it with the date, then extract the deals from the image in your own pipeline.

### How do I scrape a store locator?

Most chains publish a page per store under a directory such as /stores/ for search engines. Crawl that directory with an include pattern and you get every store's page. When a chain only has a map, call the locator for a grid of postcodes and deduplicate stores by ID.

### What retail price data matters for CPG brands?

Shelf price, promotional price, promotion mechanic and dates, pack size and unit price, per chain and region. Brands compare their range with competitors' at the same stores, so keep the store and the collection date on every record.

### How do I scrape supermarket prices across chains?

Pick one basket of comparable products, then map each item to its product page on every chain you track. Collect each chain's prices from the stores in your panel, with the store set the way that site expects it. Parse pack size into a unit price per litre, kilo or count, so a 4-pint bottle and a 2-litre bottle compare directly.

## Related use cases

- [Ecommerce web scraping](https://www.zenrows.com/use-cases/ecommerce-web-scraping.md): Product, price, stock, review and seller data from Amazon, Walmart, eBay and Shopify stores, as JSON.
- [Competitor price scraping](https://www.zenrows.com/use-cases/price-scraping.md): Competitor prices, promotions, shipping and stock per SKU from retailers and marketplaces, refreshed on your schedule.
- [Digital shelf analytics](https://www.zenrows.com/use-cases/digital-shelf-analytics.md): Search rank, share of shelf, content compliance, ratings and availability for CPG brands, per retailer and region.
- [Market research web scraping](https://www.zenrows.com/use-cases/market-research-web-scraping.md): Assortments, prices, reviews and forum talk across a whole category, for sizing studies and recurring trackers.

Start free: https://app.zenrows.com/register. Docs: https://docs.zenrows.com