JSON, JSONL, CSV or XML: which output format?

The format you write scraped data in decides how easy it is to resume a failed run, how much memory a downstream job needs, and whether you can keep nested fields. It is a small decision with a long tail.

JSONL is the default for a scrape

JSONL, sometimes called NDJSON, is one JSON object per line. That single property changes the operational picture more than it looks.

You can append. A JSON array has to be closed with ], so writing records into one as you go means either holding everything in memory until the end or rewriting the file. With JSONL you append a line per record and stop whenever you like.

A crash keeps its data. Kill a job writing a JSON array and the file is invalid, and you have nothing. Kill one writing JSONL and every completed line is still readable. On a run of a million pages this is the difference between resuming and restarting.

It streams. Downstream jobs read a line at a time with constant memory, rather than parsing a multi-gigabyte array in one go.

It splits trivially. split -l gives you shards for parallel processing.

For a scraping run of any size, this is usually the right choice.

Where the others belong

JSON suits a single response or a small complete result set, and is the natural shape for an API to return. It is a poor fit for an append-only log of scraped records.

CSV is right when data is genuinely flat and a person will open it. Its weakness is nesting: a product with a list of variants has to be flattened or serialised into a cell, and both are lossy in practice. Quoting and encoding cause more real-world breakage than they should.

XML is rarely chosen for new work but often encountered: RSS feeds, sitemaps, and older enterprise interfaces. Worth being able to read even if you never write it.

The practical arrangement

Most pipelines end up with two stages. Write JSONL during collection, because it is append-safe and resumable. Convert to CSV or a database at the end for whoever consumes it. Trying to write the final format during collection is what makes a failed run expensive.

Where Zenrows fits

Extract returns structured JSON per request, which fits either arrangement: write one line per response and you have JSONL. Batch runs a URL list as a managed job and collects results when it finishes, which removes the partial-write problem from your side entirely.