---
title: "Website content migration"
description: "Website content migration with Crawl: export every page's body, title, meta description, images and URL from a legacy CMS as Markdown or JSON."
canonical: https://www.zenrows.com/use-cases/website-content-migration
updated: 2026-10-18
---

# Website content migration

Website content migration moves every page of an existing site into a new CMS, including body copy, headings, metadata, images and the URL structure your redirects depend on. Zenrows Crawl walks the old site and returns each page as Markdown, and Fetch pulls each page's metadata as JSON, giving you an import file instead of weeks of copy and paste.

## Sample output: one page ready for import

```json
{
  "old_url": "https://www.example.com/services/commercial-roofing.aspx",
  "new_path": "/services/commercial-roofing",
  "status": 200,
  "title": "Commercial roofing services | Example Co",
  "meta_description": "Flat and low-slope roof installation and repair for warehouses and offices.",
  "canonical": "https://www.example.com/services/commercial-roofing.aspx",
  "h1": "Commercial roofing",
  "headings": ["Flat roof systems", "Repairs and inspections", "Service area"],
  "body_markdown": "## Flat roof systems\n\nWe install TPO and EPDM membranes on...",
  "word_count": 812,
  "images": [
    { "src": "/media/roof-tpo-install.jpg", "alt": "TPO membrane being installed" }
  ],
  "internal_links": ["/services/roof-repair.aspx", "/contact.aspx"],
  "template": "service-page",
  "crawled_at": "2026-10-17T10:03:51Z"
}
```

An illustrative page record; `new_path` and `template` are columns your team adds when mapping, and everything else comes from the old site.

## What a migration needs from the old site

A content migration fails on the pages nobody listed. The work starts with a full inventory and ends with a check that every old URL landed somewhere.

| Job | Data you collect | Typical sources |
|---|---|---|
| Content inventory | Every URL, status code, template type, word count | A full crawl of the live site, compared with its sitemap.xml |
| Body content export | Main copy as Markdown with headings, lists, tables and links | WordPress, Drupal, Sitecore, Adobe Experience Manager, custom ASP.NET or PHP sites |
| SEO metadata capture | Title, meta description, canonical, H1, hreflang, robots directives | The `<head>` of every page |
| Redirect mapping | Old URL, new URL, status code | The crawl inventory joined with the new site's URL plan |
| Media and file inventory | Image URLs, alt text, linked PDFs and downloads | `img` and `a` tags across every page |
| Post-launch verification | New URLs, statuses, titles and word counts compared with the old site | A second crawl of the new site after cut-over |

## Why manual content migration breaks

Copying pages by hand works for twenty pages. Past a few hundred, pages go missing, metadata gets dropped and old URLs return 404s the week after launch.

| What breaks | What it looks like | How Zenrows handles it |
|---|---|---|
| No usable export | The old CMS exports database tables full of shortcodes and layout markup, or has no export at all | Crawl reads the rendered pages, so you get the content visitors actually see |
| Pages nobody listed | Landing pages missing from the sitemap and the menu, reachable only from old blog posts | Crawl follows every internal link on the domain, and you diff its URL list against the sitemap |
| JavaScript-rendered content | Tabs, accordions and single-page app routes come back empty from a plain HTTP request | Adaptive Stealth Mode escalates to browser rendering when a page needs it and returns the page after it renders |
| Header and footer in every body | The export repeats navigation, sidebar and footer text on every page | Name the content container with a CSS selector, such as `article` or `#main`, and get only that |
| Lost SEO metadata | Titles, descriptions and canonicals vanish during import, and search traffic drops after launch | Name the head fields with CSS selectors and get them as JSON for every page, ready to map into the new CMS |
| Your own firewall | Cloudflare or a WAF in front of the old site challenges the crawler | Zenrows' stealth gets through Cloudflare and similar protections, or allowlist the crawl in your firewall |

## How to run a website content migration

1. **Inventory with Crawl.** Start a crawl at the homepage with no output format, so it returns every URL first, and compare that list with the sitemap.
2. **Export content as Markdown.** Run Crawl again with Markdown output, or fetch each URL and name the content container with a CSS selector.
3. **Capture metadata as JSON.** Fetch each URL with CSS selectors for title, description, canonical, H1 and images, and write one row per page.
4. **Map and import.** Assign each old URL a new path and template, transform the Markdown to the new CMS format and build the redirect map from the same table.
5. **Verify after launch.** Crawl the new site, join both inventories and fix every old URL that has no 301 and every page whose word count dropped.

```bash cURL
curl -X POST "https://api.zenrows.com/v1/crawls" \
  -H "X-API-Key: YOUR_ZENROWS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://www.example.com/",
    "max_items": 5000,
    "exclude_patterns": ["/search", "/cart", "/wp-admin"],
    "output_format": "markdown"
  }'
```

```python Python
# Capture SEO metadata for every URL in the inventory
# pip install requests
import csv, json, requests

EXTRACTOR = {
    "title": "title",
    "description": "meta[name=description] @content",
    "canonical": "link[rel=canonical] @href",
    "h1": "h1",
    "images": "img @src",
}

urls = [line.strip() for line in open("inventory.txt") if line.strip()]
with open("metadata.csv", "w", newline="") as out:
    writer = csv.writer(out)
    writer.writerow(["url", "title", "description", "canonical", "h1", "images"])
    for url in urls:
        data = requests.get("https://api.zenrows.com/v1/", params={
            "apikey": "YOUR_ZENROWS_API_KEY",
            "url": url,
            "mode": "auto",
            "css_extractor": json.dumps(EXTRACTOR),
        }).json()
        writer.writerow([url] + [json.dumps(data.get(k)) for k in EXTRACTOR])
```

```bash CLI
npx -y @zenrows/cli init
zenrows crawl create https://www.example.com/ --max-pages 5000 --output-format markdown --follow
zenrows crawl results CRAWL_ID --out site-export.jsonl
zenrows fetch https://www.example.com/services/commercial-roofing.aspx --output markdown
```

```text Claude Code (MCP)
# Connect once
claude mcp add zenrows -e ZENROWS_API_KEY=YOUR_ZENROWS_API_KEY -- npx -y @zenrows/mcp

# Then ask:
Use Zenrows to crawl www.example.com, then give me a content inventory as a table with each URL, its title, meta description, canonical, H1 and word count, and flag pages that are missing a meta description.
```

## Which Zenrows product fits

| If you need to | Use | Why |
|---|---|---|
| List every URL on the old site | [Crawl](/products/crawl) | Follows internal links across the domain and returns the full URL inventory |
| Export every page body as Markdown | [Crawl](/products/crawl) | Markdown output returns clean content for each page in one job |
| Capture titles, descriptions and canonicals per page | [Fetch](/products/fetch) | Name the head fields with CSS selectors and get them back as JSON |
| Get fields from pages with no consistent template | [Extract](/products/extract) | Automatic structured extraction returns content without selectors per template |
| Spot-check pages from the terminal during mapping | [CLI](/cli) | One command returns a page as Markdown to compare with the new CMS |

## What success looks like

| Measure | What good looks like | How to check it |
|---|---|---|
| URL parity | Every old URL that returned 200 has a new page, a 301 or a deliberate 410 | Join the old and new crawl inventories and list old URLs with no match |
| Redirect accuracy | Every 301 lands on the closest equivalent page in one hop | Fetch each old URL after launch and record the final URL and hop count |
| Content fidelity | Word counts on new pages match the old pages within your tolerance | Compare word counts per page pair and review the largest drops |
| Metadata retention | Titles, descriptions and canonicals are present on every migrated page | Re-run the metadata extractor on the new site and count empty fields |
| Broken links | No internal link on the new site points to a 404 | Crawl the new site and list internal links with error statuses |

## Website content migration FAQ

### What is website content migration?

Website content migration is moving the content of an existing site, its pages, posts, metadata, images and files, into a new CMS or platform. It runs alongside the design and development work and decides whether the new site launches complete. The core deliverables are a content inventory, an export in a format the new CMS can import, and a redirect map.

### How do I migrate content from one CMS to another?

Inventory the old site with a full crawl, export each page as Markdown or JSON, map every old URL to a new path and template, then import with the new CMS's API or import tool. Build the redirect map from the same mapping table, and crawl the new site after launch to confirm nothing went missing.

### How do I keep SEO during a content migration?

Capture each page's title, meta description, canonical, H1 and hreflang before you move anything, and carry them into the new CMS. Keep URLs where you can, 301 every URL you change to its closest equivalent and update internal links so they point at final URLs. Then compare both crawls after launch.

### Can I migrate a site that has no export?

Yes. Crawl reads the rendered pages, so it does not depend on database access or an export feature. That makes it the route for proprietary CMSs, agency-built sites you no longer have admin access to, and static sites whose source files were lost.

### How do I find every page on the old site?

Combine three lists: the sitemap.xml, a full crawl that follows internal links, and the URLs that received traffic in your analytics over the past year. Pages that appear in analytics but not in the crawl are often orphaned landing pages, and they need a decision before launch.

### What belongs on a website migration checklist?

Before launch: a full URL inventory, a content export, the metadata of every page, an image and file list, a URL mapping table and a redirect map built from it. At launch: the redirects live, the new sitemap submitted and internal links updated to final URLs. After launch: a crawl of the new site joined to the old inventory, a check for 404s and redirect chains, and a review of pages whose word count or metadata changed.

### Does a content migration include images and files?

Collect every image URL with its alt text and every linked PDF or download during the crawl. Download the files from those URLs into the new CMS's media library, then rewrite the references in the Markdown so the new pages point at the new locations. Alt text moves with each image reference.

### Can I export a website to Markdown?

Yes. Run Crawl with Markdown output to get the whole site as Markdown, or call Fetch with Markdown output for single pages. Markdown keeps headings, lists, links and tables, and most headless CMSs and static site generators import it directly.

## Related use cases

- [LLM web scraping for RAG and agents](https://www.zenrows.com/use-cases/llm-web-scraping.md): Live pages as Markdown or JSON for RAG pipelines, grounding and agent tool calls.
- [Product data scraping](https://www.zenrows.com/use-cases/product-data-scraping.md): Whole product catalogs, variants, specs, images and sellers from marketplaces, normalized into one feed.
- [AI training data](https://www.zenrows.com/use-cases/ai-training-data.md): Web corpora for pre-training and fine-tuning, collected as Markdown JSONL with a source URL on every document.

Start free: https://app.zenrows.com/register. Docs: https://docs.zenrows.com