---
title: "Healthcare data scraping"
description: "Healthcare data scraping for clinical trial registries, FDA recalls, hospital price transparency files and medical research, returned as Markdown or JSON."
canonical: https://www.zenrows.com/use-cases/healthcare-data-scraping
updated: 2026-10-18
---

# Healthcare data scraping

Healthcare data scraping collects public health information from the web, such as clinical trial registries, drug and device recalls, hospital price transparency pages, research papers and health content. Zenrows renders those pages and returns Markdown or structured JSON. Crawl maps a registry or agency section, and Batch refreshes thousands of records on a schedule.

## Sample output: one trial record

```json
{
  "url": "https://registry.example.org/study/EX-0418",
  "registry_id": "EX-0418",
  "title": "Once-weekly therapy in adults with type 2 diabetes",
  "condition": ["Type 2 diabetes"],
  "intervention": { "type": "drug", "name": "Example compound EXB-21" },
  "phase": "Phase 3",
  "status": "recruiting",
  "sponsor": "Example Biotech",
  "enrollment_target": 1240,
  "start_date": "2026-03-02",
  "primary_completion_date": "2028-06-30",
  "primary_endpoint": "Change in HbA1c from baseline at week 52",
  "countries": ["US", "DE", "ES", "JP"],
  "site_count": 186,
  "last_update_posted": "2026-10-14",
  "fetched_at": "2026-10-16T05:30:00Z"
}
```

An illustrative registry record as JSON. Store `last_update_posted` next to your own fetch time so each refresh shows what moved.

## What healthcare data teams collect

Healthcare web data is mostly public by design: registries, regulators and hospitals publish it so patients, clinicians and researchers can find it. The work is in collecting it from dozens of differently built sites and keeping it current.

| Job | Data you collect | Typical sources |
|---|---|---|
| Clinical trial tracking | Status, phase, enrollment, sites, sponsor, endpoints, dates | ClinicalTrials.gov, EU CTIS, ISRCTN, WHO ICTRP, national registries |
| Recall and safety alerts | Product, lot numbers, reason, recall class, distribution, date | FDA recalls and enforcement reports, MHRA alerts, Health Canada Recalls and Safety Alerts |
| Hospital price transparency | Standard charges, payer-negotiated rates, CPT and DRG codes | Hospital machine-readable files listed in each site's `cms-hpt.txt`, shoppable services pages |
| Research monitoring | Title, authors, abstract, journal, date, DOI | PubMed, medRxiv, Google Scholar, journal tables of contents |
| Medical supply pricing | SKU, price, pack size, availability | Medline, Henry Schein and other distributor catalogs |
| Health content for search and AI | Condition pages, guidance and public health updates as clean text | WHO, CDC, NHS, Mayo Clinic |

## Why healthcare scraping breaks

Each registry, regulator and hospital system builds its site differently, and the records you care about sit several clicks deep. Most failures come from one of these.

| What breaks | What it looks like | How Zenrows handles it |
|---|---|---|
| Search results built in JavaScript | The registry page returns an empty shell; results and filters load afterwards | Adaptive Stealth Mode escalates to browser rendering when the page needs it, and can wait for the results element to load |
| Protected hospital and publisher sites | A challenge page or a 403 instead of the price page or article | Zenrows' stealth gets through Cloudflare, Akamai, Imperva and similar protections and returns the price page or article |
| Records spread across thousands of URLs | Recall notices and trial records have no single index page you can read | Crawl follows the section and keeps only URLs matching the URL patterns you set |
| Long pages with tabs and accordions | Navigation, cookie banners and collapsed sections bury the record text | Markdown output returns the main content, and the tables output filter pulls the tables |
| Quiet updates | A status or completion date changes on the same URL with no notice | Batch re-fetches the URL list on schedule and you diff the fields |

## How to scrape healthcare data with Zenrows

1. **Map the source.** Point Crawl at a registry or agency section, for example an FDA recalls listing, and keep only record URLs with a URL pattern to keep.
2. **Fetch each record.** Request each URL with Adaptive Stealth Mode and Markdown output, or name the fields with CSS selectors when you want structured data.
3. **Key every record.** Store the trial ID, recall number or charge code with the source URL, so later runs update the same row.
4. **Refresh and diff.** Send the URL list to Batch daily for recalls and weekly for trials, and compare each run with the last.

```bash cURL
curl -X POST "https://api.zenrows.com/v1/crawls" \
  -H "X-API-Key: YOUR_ZENROWS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "RECALLS_LISTING_URL",
    "max_items": 1000,
    "max_pages": 1200,
    "include_patterns": ["/safety/recalls-market-withdrawals-safety-alerts/"],
    "output_format": "markdown"
  }'
# Returns a crawl_id. Read the pages with GET /v1/crawls/{crawl_id}
```

```python Python
# pip install requests
import requests

params = {
    "apikey": "YOUR_ZENROWS_API_KEY",
    "url": TRIAL_RECORD_URL,
    "mode": "auto",  # Adaptive Stealth Mode
    "response_type": "markdown",
}
response = requests.get("https://api.zenrows.com/v1/", params=params)
print(response.text)  # the trial record as clean Markdown
```

```bash CLI
npx -y @zenrows/cli init
# One trial record as Markdown
zenrows fetch TRIAL_RECORD_URL --output markdown
# The tables from a recall notice as JSON
zenrows extract RECALL_NOTICE_URL --outputs tables
```

```text Claude Code (MCP)
# Connect once
claude mcp add zenrows -e ZENROWS_API_KEY=YOUR_ZENROWS_API_KEY -- npx -y @zenrows/mcp

# Then ask:
Use Zenrows to read the five newest notices on the FDA recalls, market withdrawals and safety alerts page and return one table with product, lot numbers, reason for recall, distribution area and recall date.
```

## Which Zenrows product fits

| If you need to | Use | Why |
|---|---|---|
| Read one trial record, recall notice or guidance page | [Fetch](/products/fetch) | Renders the page and returns Markdown, HTML or the fields you name |
| Find every record URL in a registry or agency section | [Crawl](/products/crawl) | Follows links on one domain and filters URLs by pattern |
| Refresh thousands of records on a schedule | [Batch](/products/batch) | Sends every trial and recall URL as one job, with results by webhook |
| Summarise recalls or new trials with an assistant | [MCP Server](/mcp) | Claude, Cursor and other MCP clients call Zenrows to read the pages |

## What success looks like

| Measure | What good looks like | How to check it |
|---|---|---|
| Registry coverage | Every trial that matches your condition and phase filters is in your table | Compare your count with the registry's own result count for the same query |
| Status lag | Status and date changes appear within one refresh cycle | Compare `last_update_posted` with your fetch time on each record |
| Recall lead time | A new recall notice is in your system the day it is posted | Log publication date against ingestion date per notice |
| Price file currency | You hold the current machine-readable file for every tracked hospital | Check the file URL and date in `cms-hpt.txt` on every run |
| Field completeness | Phase, status, sponsor and enrollment are filled on nearly every trial | Report empty fields per registry after each run |
| Failed fetches per run | Nearly every record URL in a run returns content on the first pass | Count URLs that returned an error or an empty record, per registry and run |

## Healthcare data scraping FAQ

### Is healthcare data scraping legal?

Registries, recall databases, hospital price files and published research are public so that people can use them, and collecting them is common in research and market analysis. Keep away from patient records: HIPAA governs protected health information held by covered entities and their business associates, and public registry pages are not that. Investigator names and contact details are still personal data under GDPR, so collect only what you need. Respect each registry's and publisher's terms, and have your legal team review the project.

### What healthcare data can I collect from the web?

Trial records (status, phase, sites, endpoints), recall and safety notices, hospital standard charges and negotiated rates, research metadata and abstracts, medical supply catalogs, and public health guidance. Each source has its own record page, so most projects pair a listing page that yields URLs with a detail page that yields the fields.

### Should I use the ClinicalTrials.gov API instead of scraping?

Use an official API where it gives you the fields you need. ClinicalTrials.gov and openFDA both publish one. Many national registries, sponsor pipeline pages, hospital sites and regulator notices outside the US have no API, and that is where Zenrows does the collection, in the same format as the rest of your data.

### How do I find a hospital's price transparency file?

US hospitals list the location of their machine-readable standard charges file in a cms-hpt.txt file at the root of their website, and link to a price transparency page from the homepage. Fetch the text file on each run to read the current file URL, then download the file when the URL or date changes.

### How often should healthcare data be refreshed?

Match the refresh to how fast the source moves. Recall and safety notices are worth checking daily. Trial records change when a sponsor files an update, so a weekly pass with a diff catches most status and date changes. Price transparency files are updated at least yearly, so a monthly check on cms-hpt.txt is enough.

### How do I scrape medical research papers and abstracts?

Collect titles, authors, abstracts, journal, date and DOI from PubMed, medRxiv or a journal's table of contents. Crawl a table of contents or a search result list to get article URLs, then fetch each one as Markdown or name the fields with CSS selectors. Key every record by DOI so preprints and their published versions merge into one row.

### Can I use scraped medical content to train an AI model?

Licensing is decided per source. Guidance from WHO, NHS and medical publishers comes with each publisher's own terms, and only part of PubMed Central is licensed for reuse. Check the licence of each source before building a training set, and keep the source URL with every document so you can show where it came from.

## Related use cases

- [Healthcare provider data](https://www.zenrows.com/use-cases/healthcare-provider-data.md): Clinician and facility records from payer directories and hospital sites, matched on NPI and re-verified on schedule.
- [Pharmaceutical data scraping](https://www.zenrows.com/use-cases/pharmaceutical-data.md): Drug prices by pharmacy, formulary tiers, FDA and EMA approvals and patient-reported signals, tracked over time.
- [Insurance data scraping](https://www.zenrows.com/use-cases/insurance-data.md): Competitor quotes, policy terms, rate filings and property risk data, captured per test profile.
- [Web scraping for research](https://www.zenrows.com/use-cases/web-scraping-for-research.md): Article metadata, forum posts, statistics and news text collected for studies, with a reproducible fetch log.
- [AI training data](https://www.zenrows.com/use-cases/ai-training-data.md): Web corpora for pre-training and fine-tuning, collected as Markdown JSONL with a source URL on every document.

Start free: https://app.zenrows.com/register. Docs: https://docs.zenrows.com