---
title: "Pharmaceutical data scraping"
description: "Pharmaceutical data from pharmacy prices, insurer formularies, FDA and EMA approvals and patient forums. Drug price data across pharmacies as JSON."
canonical: https://www.zenrows.com/use-cases/pharmaceutical-data
updated: 2026-10-18
---

# Pharmaceutical data scraping

Pharmaceutical data scraping collects drug prices, formulary placement, regulatory approvals and patient-reported signals from public websites. Zenrows fetches pharmacy product pages, formulary tools and FDA or EMA records, in each country where prices differ, and returns structured JSON. Batch reruns the full price grid, so you see every change by pharmacy, pack size and date.

## Sample output: one drug across pharmacies

```json
{
  "drug": "atorvastatin",
  "strength": "20 mg",
  "form": "tablet",
  "ndc": "12345-678-90",
  "quantity": 30,
  "market": "US",
  "observed_at": "2026-10-16",
  "prices": [
    { "pharmacy": "Online pharmacy A", "price": 14.20, "type": "cash", "in_stock": true },
    { "pharmacy": "Retail chain B", "price": 18.99, "type": "cash", "in_stock": true },
    { "pharmacy": "Discount program C", "price": 9.85, "type": "coupon", "in_stock": true }
  ],
  "currency": "USD",
  "formulary": {
    "plan": "Example Health Plan 2026",
    "tier": 1,
    "prior_authorization": false,
    "quantity_limit": "90 per 90 days"
  }
}
```

An illustrative price observation joined to one plan's formulary entry. Keeping the NDC, strength and quantity on every row is what makes prices comparable across pharmacies.

## What pharmaceutical data teams collect

Pharma teams use web data in two directions: outward to the market (prices, coverage, competitors) and inward to safety (what patients say about a product). The sources barely overlap.

| Job | Data you collect | Typical sources |
|---|---|---|
| Drug price monitoring | Cash price, coupon price, pack size, stock status | GoodRx, Cost Plus Drugs, CVS, Walgreens, Amazon Pharmacy, DocMorris, Shop Apotheke |
| Formulary and coverage tracking | Tier, prior authorization, step therapy, quantity limits | Insurer and PBM formulary search tools and documents, Medicare Plan Finder |
| Regulatory approvals and labels | Approval date, indication, label changes, generic and biosimilar entry | Drugs@FDA, Orange Book, Purple Book, EMA EPARs, Health Canada Drug Product Database |
| Competitor pipeline | Phase changes, readouts, partnerships | Company pipeline pages, press release wires, BioSpace, trial registries |
| Pharmacovigilance signals | Reported side effects, product, dosage, date of post | Drugs.com and WebMD reviews, patient forums, health communities on Reddit |
| OTC product tracking | Price, rating, review count, availability | Amazon, Walmart, Boots, pharmacy chain sites |

## Why pharma data collection breaks

A drug price is only meaningful with its strength, quantity, pharmacy and location attached, and each pharmacy site shows those differently.

| What breaks | What it looks like | How Zenrows handles it |
|---|---|---|
| Price depends on location and quantity | The same drug shows a different price per ZIP code, pharmacy and 30 or 90 count | Build one URL per grid cell and use geotargeting to request each market from its own country |
| Pharmacy and discount sites block scripts | Cloudflare challenges, or a page whose price element is empty | Adaptive Stealth Mode escalates to browser rendering and Zenrows' stealth only when a page needs it, and returns the page a shopper sees |
| Formularies hide behind search tools | Tier and restrictions appear only after choosing plan year and typing a drug | Browser runs the search; the links output filter lists the formulary documents on a plan page |
| One drug, many names | Brand, generic, salt form and pack description differ by site | Name NDC, strength and form with CSS selectors and normalise on your side |
| Forum volume is mostly noise | Thousands of posts a week, most unrelated to a product | Markdown output returns clean post text ready for classification |

## How to collect pharmaceutical data with Zenrows

1. **Build the price grid.** List each drug, strength, quantity, pharmacy and location you track. Each cell becomes one URL.
2. **Name the fields once.** Test one product page, naming price, pack size, NDC and stock with CSS selectors, or use Extract to skip selectors.
3. **Run the grid as a job.** Submit every URL to Batch daily and store each result with its grid cell and date.
4. **Pull regulatory pages as text.** Fetch approval, label and EPAR pages as Markdown from the CLI and diff them against the last version.

```python Python
# pip install requests
import json
import requests

# Example selectors: match them to the pharmacy's product page
fields = {
    "name": "h1",
    "price": ".price",
    "pack_size": ".pack-size",
    "ndc": ".ndc",
    "stock": ".availability",
}
params = {
    "apikey": "YOUR_ZENROWS_API_KEY",
    "url": "PHARMACY_PRODUCT_URL",
    "mode": "auto",
    "proxy_country": "us",
    "css_extractor": json.dumps(fields),
}
response = requests.get("https://api.zenrows.com/v1/", params=params)
print(response.json())
```

```bash cURL
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
  -H "X-API-Key: YOUR_ZENROWS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "regular",
    "status": "closed",
    "zenrows_params": { "mode": "auto", "extract": "auto" },
    "tasks": [
      { "url": "PHARMACY_A_PRODUCT_URL", "external_id": "atorvastatin-20mg-30-pharmacy-a" },
      { "url": "PHARMACY_B_PRODUCT_URL", "external_id": "atorvastatin-20mg-30-pharmacy-b" }
    ]
  }'
```

```bash CLI
npx -y @zenrows/cli init
# Approval and EPAR pages as Markdown, ready to diff against the last version
zenrows fetch EPAR_PAGE_URL --output markdown --out epar-today.md
# Formulary documents linked from a plan page
zenrows extract PLAN_PAGE_URL --outputs links
```

```text Claude Code (MCP)
# Connect once
claude mcp add zenrows -e ZENROWS_API_KEY=YOUR_ZENROWS_API_KEY -- npx -y @zenrows/mcp

# Then ask:
Use Zenrows to look up atorvastatin 20 mg, 30 tablets, on GoodRx and Cost Plus Drugs
for ZIP code 10001, and return each pharmacy's name and price as a table.
```

## Which Zenrows product fits

| If you need to | Use | Why |
|---|---|---|
| Read a pharmacy product page or a label page | [Fetch](/products/fetch) | One request returns the rendered page from protected pharmacy sites |
| Get price and pack data without maintaining selectors | [Extract](/products/extract) | Returns structured fields that adapt when a pharmacy redesigns |
| Price the whole grid every day | [Batch](/products/batch) | Thousands of drug and pharmacy URLs as one job, with an `external_id` per cell |
| Pull approval and EPAR pages into a script or pipeline | [CLI](/cli) | One command returns a page as Markdown |
| Search a formulary tool by plan and drug name | [Browser](/products/browser) | A real browser session for forms and dropdowns |

## What success looks like

| Measure | What good looks like | How to check it |
|---|---|---|
| Grid coverage | Every drug, strength, quantity and pharmacy cell has a price for the day | Divide filled cells by planned cells after each run |
| Normalisation rate | Nearly every observation maps to an NDC or your own product ID | Count rows left unmapped per pharmacy |
| Approval lead time | New FDA approvals and EMA opinions reach your system the day they are posted | Compare publication date with ingestion date |
| Formulary change capture | Tier and restriction changes are caught in the cycle they appear | Diff each plan's entries run over run |
| Price freshness | Every price in the grid was collected in the latest daily run | Count cells whose newest observation is older than one day |

## Pharmaceutical data FAQ

### What is pharmaceutical data?

In this context it is public, web-published information about medicines: retail and discount prices, formulary placement and restrictions, approval and labelling records, pipeline announcements, and what patients and clinicians say about a product. Commercial, market access and safety teams each use a different slice of it.

### Where can I get drug price data?

Retail and discount prices are on pharmacy and discount program websites, such as GoodRx, Cost Plus Drugs, CVS, Walgreens and Amazon Pharmacy in the US, or DocMorris and Shop Apotheke in Europe. CMS also publishes NADAC, a weekly survey of pharmacy acquisition costs, as a free download. Scraping the retail pages gives you what a patient pays at the counter.

### Is it legal to scrape pharmaceutical data and patient forums?

Prices, formularies and regulatory records are published for public use. Patient posts are different: health information about an identifiable person is special-category data under GDPR, so collect post text without usernames, keep only what your purpose needs, and check each forum's terms. HIPAA does not cover what patients post themselves, but other laws may. Ask your legal team about your specific use.

### Do posts I find on social media count as adverse event reports?

That is a decision for your pharmacovigilance team. EMA's good pharmacovigilance practice guidance covers screening digital media, and companies typically document how a post found by monitoring is reviewed and when it is followed up. Collecting the posts with the URL and date makes that review possible.

### How do I track FDA and EMA approvals?

Drugs@FDA, the Orange Book and openFDA offer downloads and an API for approval records. Labels, press announcements, EMA committee meeting highlights and EPAR pages change as web pages, so teams fetch them as Markdown on a schedule and diff each version against the last.

### What is formulary data, and how is it collected?

Formulary data lists which drugs a health plan covers, at which tier, and with which restrictions such as prior authorization, step therapy or quantity limits. Plans publish it as searchable tools and as PDF or spreadsheet documents on their websites. Collect the documents from each plan page, run the search tools in a browser session for plans that only offer search, and diff each plan's entries against the last run to catch tier changes.

### Can I compare drug prices across countries?

Yes. Use geotargeting to request each market from its own country, for example Germany for German online pharmacies or the UK for UK chains, so each site returns local prices and currency. Store the market and currency on every row, and convert in your own pipeline so the original price is never lost.

## Related use cases

- [Healthcare data scraping](https://www.zenrows.com/use-cases/healthcare-data-scraping.md): Clinical trials, recalls, hospital prices and research papers from public registries and agency sites, kept current.
- [Healthcare provider data](https://www.zenrows.com/use-cases/healthcare-provider-data.md): Clinician and facility records from payer directories and hospital sites, matched on NPI and re-verified on schedule.
- [Competitor price scraping](https://www.zenrows.com/use-cases/price-scraping.md): Competitor prices, promotions, shipping and stock per SKU from retailers and marketplaces, refreshed on your schedule.
- [Sentiment analysis data](https://www.zenrows.com/use-cases/sentiment-analysis-data.md): Review text with ratings, forum threads and comments, cleaned and labeled for sentiment and NLP models.
- [Insurance data scraping](https://www.zenrows.com/use-cases/insurance-data.md): Competitor quotes, policy terms, rate filings and property risk data, captured per test profile.

Start free: https://app.zenrows.com/register. Docs: https://docs.zenrows.com