---
title: "Twitter (X) scraper"
description: "Scrape public X (Twitter) profiles and posts as JSON: bio, followers, post count, post text, replies, reposts, likes and views. One request from Python or MCP."
canonical: https://www.zenrows.com/scrape/twitter
updated: 2026-10-20
---

# Twitter (X) scraper

The Zenrows Twitter (X) scraper turns a public x.com profile into structured JSON: display name, username, bio, location, verified flag, join date, follower, following and post counts, plus recent posts with text, date, replies, reposts, likes and views. Fetch in Adaptive Stealth Mode opens the page, and Extract returns the fields. Run it from an API call, the CLI or an AI agent.

- Site: x.com
- Returns: Profiles, follower counts, posts and engagement

## Run it: one X profile in, JSON out

```json Input
{
  "url": "https://x.com/SpaceX",
  "mode": "auto",
  "extract": "auto"
}
```

```json Output
{
  "profile": {
    "username": "SpaceX",
    "display_name": "SpaceX",
    "bio": "SpaceX designs, manufactures and launches the world’s most advanced rockets and spacecraft",
    "location": "Earth",
    "joined_date": "April 2009",
    "is_verified": true,
    "profile_url": "https://x.com/SpaceX",
    "avatar_url": "https://pbs.twimg.com/profile_images/1697749409851985920/HbrI04tM_400x400.jpg"
  },
  "stats": {
    "follower_count": "41.9M",
    "following_count": "127",
    "post_count": "11.9K"
  },
  "posts": [
    {
      "text": "Teams encapsulated 32 payloads ahead of Falcon 9’s launch of the Bandwagon-5 rideshare mission from Florida → spacex.com/launches/bandw…",
      "posted_date": "Oct 17",
      "post_url": "https://x.com/SpaceX/status/2111482288135446909",
      "reply_count": "408",
      "repost_count": "879",
      "like_count": "7.3K",
      "view_count": "1M"
    }
  ]
}
```

Sample output for an X profile, built from a live request to x.com/SpaceX on Oct 20, 2026, trimmed for length. A field is returned when the page shows it.

## What data can you scrape from X (Twitter)?

A public profile page carries the account's identity, its counters and its latest posts. These are the fields teams use most, with the path where each lives in the JSON.

| Field | What it holds | Example |
|---|---|---|
| `profile.username` | The handle, without the @ | `SpaceX` |
| `profile.display_name` | The name shown above the handle | `SpaceX` |
| `profile.bio` | The profile description | `SpaceX designs, manufactures and launches the world’s most advanced rockets and spacecraft` |
| `profile.location` | The location the account entered | `Earth` |
| `profile.joined_date` | Month and year the account was created | `April 2009` |
| `profile.is_verified` | Whether the account shows a verified badge | `true` |
| `profile.avatar_url` | Profile picture, 400 by 400 | `https://pbs.twimg.com/profile_images/…` |
| `profile.banner_url` | The header image | `https://pbs.twimg.com/profile_banners/…` |
| `stats.follower_count` | Followers, as X displays them | `41.9M` |
| `stats.following_count` | Accounts it follows | `127` |
| `stats.post_count` | Total posts on the account | `11.9K` |
| `posts[].text` | The text of each recent post, with links and mentions | `We will post our Q3 2026 financial…` |
| `posts[].posted_date` | The date shown on the post | `Oct 16` |
| `posts[].post_url` | Permanent link, with the post ID | `https://x.com/SpaceX/status/2111072697690317011` |
| `posts[].reply_count` | Replies | `387` |
| `posts[].repost_count` | Reposts | `1.2K` |
| `posts[].like_count` | Likes | `10K` |
| `posts[].view_count` | Views | `1.4M` |

## How to scrape X (Twitter) with Zenrows

1. **Create a free Zenrows account** and copy your API key from the dashboard.
2. **Pick the profile URL.** `https://x.com/<handle>` is the page a logged-out visitor sees, with the profile, its counters and the latest posts. Old `twitter.com` links redirect to the same page.
3. **Send it with the default configuration**: Adaptive Stealth Mode to open the page and Extract to return the fields. X renders profiles in the browser, and Adaptive Stealth Mode adds rendering when the page needs it.
4. **Read `parsed`** from the response. Counters come back as X shows them (`41.9M`, `11.9K`), so convert them to numbers in your code.
5. **Run your list of handles in Batch** once one profile comes back the way you want.

```python Python
# pip install requests
import requests

params = {
    "apikey": "YOUR_ZENROWS_API_KEY",
    "url": "https://x.com/SpaceX",
    "mode": "auto",          # Adaptive Stealth Mode
    "extract": "auto",       # structured JSON
}
response = requests.get("https://api.zenrows.com/v1/", params=params, timeout=180)
response.raise_for_status()

data = response.json()["parsed"]

def to_number(value):
    """Turn X's short counters ("41.9M", "11.9K") into integers."""
    if not value:
        return None
    value = value.replace(",", "")
    scale = {"K": 1_000, "M": 1_000_000, "B": 1_000_000_000}.get(value[-1], 1)
    return int(float(value.rstrip("KMB")) * scale)

print(data["profile"]["username"], to_number(data["stats"]["post_count"]))
for post in data.get("posts", []):
    print(post.get("posted_date"), to_number(post.get("like_count")), post.get("text", "")[:80])
```

```javascript Node.js
// Node 18+ (built-in fetch)
const params = new URLSearchParams({
  apikey: "YOUR_ZENROWS_API_KEY",
  url: "https://x.com/SpaceX",
  mode: "auto",
  extract: "auto",
});

const response = await fetch(`https://api.zenrows.com/v1/?${params}`, {
  signal: AbortSignal.timeout(180_000),
});
if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);

const { parsed } = await response.json();
console.log(parsed.profile.display_name, parsed.profile.bio, parsed.stats);
```

```bash cURL
curl "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fx.com%2FSpaceX&mode=auto&extract=auto"

# Only the fields you need
curl "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fx.com%2FSpaceX&mode=auto&extract=auto&extract_fields=profile.username%2Cprofile.bio%2Cprofile.is_verified%2Cstats.post_count"
```

```python Python SDK
# pip install zenrows
from zenrows import ZenRowsClient

client = ZenRowsClient("YOUR_ZENROWS_API_KEY")
for handle in ["SpaceX", "Tesla", "OpenAI"]:
    response = client.get(f"https://x.com/{handle}", params={"mode": "auto", "extract": "auto"})
    profile = response.json()["parsed"]["profile"]
    print(profile["username"], profile.get("location"), profile.get("joined_date"))
```

## Input

The scraper takes one x.com URL per request, plus the options below. The same parameters go into every task of a Batch job, so the input for a thousand accounts is a list of profile URLs with these settings.

| Parameter | Type | Default | Description |
|---|---|---|---|
| `url` | string | required | A public X profile, `https://x.com/<handle>`. Links on twitter.com work too |
| `mode` | string | `auto` | Adaptive Stealth Mode: starts with the lightest setup and adds rendering and protected-site handling only when the page needs them |
| `extract` | string | `auto` | Returns the page's fields as JSON in `parsed`, next to the raw `html` |
| `extract_fields` | string | all fields | Comma-separated paths to keep, such as `profile.username,stats.post_count` |
| `proxy_country` | string | any | Two-letter country code. X shows dates and some labels in the visitor's language |
| `wait_for` | string | none | A CSS selector to wait for before the page is read, such as the post timeline |
| `session_id` | integer | none | Reuse the same session across several requests, up to 10 minutes |

```json Batch input
{
  "type": "regular",
  "status": "closed",
  "zenrows_params": {
    "mode": "auto",
    "extract": "auto"
  },
  "tasks": [
    { "url": "https://x.com/SpaceX", "external_id": "spacex" },
    { "url": "https://x.com/Tesla", "external_id": "tesla" },
    { "url": "https://x.com/OpenAI", "external_id": "openai" }
  ]
}
```

## Output

Every response is JSON with two keys: `html`, the page as X served it, and `parsed`, the structured record. A profile page returns three groups: `profile`, `stats` and `posts`. The block below is built from a live request to x.com/SpaceX on Oct 20, 2026, with every value as the page showed it.

```json Profile page
{
  "profile": {
    "username": "SpaceX",
    "display_name": "SpaceX",
    "bio": "SpaceX designs, manufactures and launches the world’s most advanced rockets and spacecraft",
    "location": "Earth",
    "joined_date": "April 2009",
    "is_verified": true,
    "profile_url": "https://x.com/SpaceX",
    "avatar_url": "https://pbs.twimg.com/profile_images/1697749409851985920/HbrI04tM_400x400.jpg",
    "banner_url": "https://pbs.twimg.com/profile_banners/34743251/1681251194/1500x500"
  },
  "stats": {
    "follower_count": "41.9M",
    "following_count": "127",
    "post_count": "11.9K"
  },
  "posts": [
    {
      "text": "Teams encapsulated 32 payloads ahead of Falcon 9’s launch of the Bandwagon-5 rideshare mission from Florida → spacex.com/launches/bandw…",
      "posted_date": "Oct 17",
      "post_url": "https://x.com/SpaceX/status/2111482288135446909",
      "image_url": "https://pbs.twimg.com/media/HU19RN3XsAAZapQ?format=webp&name=large",
      "reply_count": "408",
      "repost_count": "879",
      "like_count": "7.3K",
      "view_count": "1M"
    },
    {
      "text": "We will post our Q3 2026 financial and operational results on October 20, 2026. We will host a live audio-only webcast at 3:30 p.m. CT the same day → ir.spacex.com/updates/releas…",
      "posted_date": "Oct 16",
      "post_url": "https://x.com/SpaceX/status/2111072697690317011",
      "image_url": "https://pbs.twimg.com/media/HUwJWe2a4AAUBnd?format=webp&name=large",
      "reply_count": "387",
      "repost_count": "1.2K",
      "like_count": "10K",
      "view_count": "1.4M"
    }
  ]
}
```

Counters keep X's own format, so `7.3K` and `1M` are strings. The `to_number` helper in the Python example turns them into integers before you load them into a database. Need the page for an LLM? Ask for Markdown and the profile and posts come back as readable text.

## Scrape X from your coding agent

Claude Code, Cursor, Codex and any MCP client can read X profiles from a prompt.

The agent sends the same Fetch and Extract request a developer would, so it reads the JSON above. Connect the Zenrows MCP server once, or give the agent the CLI and let it run commands in the terminal.

```bash Connect the MCP server (Claude Code)
claude mcp add zenrows -e ZENROWS_API_KEY=YOUR_ZENROWS_API_KEY -- npx -y @zenrows/mcp
```

```json MCP config (Cursor, Claude Desktop, Windsurf)
{
  "mcpServers": {
    "zenrows": {
      "command": "npx",
      "args": ["-y", "@zenrows/mcp"],
      "env": { "ZENROWS_API_KEY": "YOUR_ZENROWS_API_KEY" }
    }
  }
}
```

```bash Or run the CLI
npx -y @zenrows/cli init
zenrows extract "https://x.com/SpaceX"
```

```text Then ask your agent
Use Zenrows to open these X profiles and build a table with the
handle, follower count, post count and the views of each account's
latest post, sorted by followers:
https://x.com/SpaceX
https://x.com/Tesla
https://x.com/OpenAI
```

## Scrape X at scale with Batch and Crawl

**Batch** runs a list of profile URLs as one job, in parallel, with your own ID on each result. Use it to snapshot follower counts and the latest posts of a few thousand accounts every day, and to keep a history that X itself does not show.

**Crawl** finds post URLs for you. Start it on a profile, keep the links that contain the account's `/status/` path, and it returns each post page it reaches. Use it when you want the post pages linked from the profile, each opened as its own page.

```bash Batch (cURL)
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
  -H "X-API-Key: YOUR_ZENROWS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "regular",
    "status": "closed",
    "zenrows_params": { "mode": "auto", "extract": "auto" },
    "tasks": [
      { "url": "https://x.com/SpaceX", "external_id": "spacex" },
      { "url": "https://x.com/Tesla", "external_id": "tesla" }
    ]
  }'

# Read the results when the job finishes
curl "https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful" \
  -H "X-API-Key: YOUR_ZENROWS_API_KEY"
```

```bash Batch (CLI)
# handles.jsonl: one {"url": "https://x.com/...", "external_id": "...", "extract": "auto"} per line
zenrows batch create handles.jsonl --wait
zenrows batch results <job-id> --out x-profiles.jsonl
```

```bash Crawl (cURL)
curl -X POST "https://api.zenrows.com/v1/crawls" \
  -H "X-API-Key: YOUR_ZENROWS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://x.com/SpaceX",
    "depth": 1,
    "max_pages": 100,
    "include_patterns": ["/SpaceX/status/"]
  }'
```

**Scheduled jobs** run the same Batch job on a timetable, on Zenrows' side: every few hours, once a day, at set times on chosen days, or once at a future date. Each run keeps its own results, a webhook can tell your app when a run finishes, and you can pause or resume the schedule at any time, with no cron server to keep alive.

```bash Scheduled job (cURL)
# Runs every day on Zenrows' side; each run keeps its own results
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
  -H "X-API-Key: YOUR_ZENROWS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
      "type": "scheduled",
      "status": "closed",
      "schedule": {
        "rate": {
          "every": 1,
          "unit": "day"
        }
      },
      "zenrows_params": {
        "mode": "auto",
        "extract": "auto"
      },
      "tasks": [
        {
          "url": "https://x.com/SpaceX",
          "external_id": "twitter"
        }
      ]
    }'

# Or at set times: "schedule": { "calendar": { "times_of_day": ["09:00"],
#   "cadence": { "weekly": { "days": ["mon", "thu"] } } }, "timezone": "Europe/Madrid" }
```

## What teams do with X data

| Use case | Data they pull | Who runs it |
|---|---|---|
| Influencer discovery and vetting | Bio, follower count, verified flag, views per post | Influencer marketing and partnerships teams |
| Competitor social tracking | Posts, posting cadence and engagement of rival accounts | Social media managers and brand teams |
| Account growth history | Daily follower, following and post counts per handle | Analysts and agencies reporting on social |
| Campaign and launch measurement | Replies, reposts, likes and views on the posts of a launch week | Marketing and communications teams |
| Public sentiment and research | Post text from official, media and expert accounts, with dates | Researchers, journalists and policy analysts |
| AI agents and RAG | The profile and latest posts as Markdown | AI and data teams |

## Tips for scraping X (Twitter)

- **Use the handle as your key.** `https://x.com/<handle>` is the stable URL of a profile. Handles are case-insensitive, so lowercase them before you dedupe.
- **Convert the counters yourself.** X shows `41.9M` and `11.9K`, which are rounded. Store the raw string next to the number you derive so later readers know the precision.
- **Keep post IDs, not dates.** `posted_date` reads `Oct 16` for this year and adds the year for older posts. The number in `/status/<id>` is unique and sorts in time order.
- **Snapshot on a schedule.** Views and likes keep rising for days after a post goes up. Request the same profiles at the same hour each day to compare like with like.
- **Expect pinned posts first.** An account can pin an older post above the timeline, so sort `posts[]` by ID when you need them newest first.
- **Read the links in the text.** X shortens URLs in the post body (`spacex.com/launches/bandw…`). Keep them as they are, or request each one to resolve its destination.

## Frequently asked questions

### Is it legal to scrape Twitter (X)?

Collecting public X data, such as profile details, follower counts and public posts that anyone can see without logging in, is common for research and social monitoring. Profiles of individuals are personal data under laws such as the GDPR and the CCPA, so collect only what you have a legitimate reason to use and do not build profiles of private people. Check X's terms of service for your use case and ask a lawyer if you are unsure.

### Is there an official X (Twitter) API?

Yes. The X API is sold in paid access levels, and reading posts at volume needs a paid plan with monthly caps on how many posts you can pull. It suits apps that post or act on behalf of an X account. Teams that only need public profile pages, as a logged-out visitor sees them, use a scraper instead.

### What X pages can I scrape?

Public profile pages at https://x.com/<handle> , which include the profile, its counters and the latest posts with their engagement. Content behind an X login, protected accounts and direct messages stay out of scope.

### Can I scrape tweets and their engagement?

Yes. The profile page returns the account's recent posts with text, date, permanent link, image, replies, reposts, likes and views. Collect a list of accounts in Batch to track engagement across many handles at once.

### Can I scrape follower counts?

Yes, the follower, following and post totals of any public profile. The list of who follows an account is only shown to logged-in users, so it stays out of scope.

### How do I scrape Twitter with Python?

Send a GET request to https://api.zenrows.com/v1/ with your API key, the x.com URL, Adaptive Stealth Mode and Extract, as in the Python example above. The response's parsed key holds profile , stats and posts as a dictionary. The zenrows package on PyPI wraps the same request.

### How many X profiles can I scrape?

Send one URL per request, or a whole list as a Batch job that runs the URLs in parallel and keeps your own ID on each result. One Batch job accepts more than 10,000 URLs, inline or as a CSV file.

### Can AI agents scrape X with Zenrows?

Yes. Connect the Zenrows MCP server to Claude, Cursor, Codex or any MCP client, or give your agent the Zenrows CLI. Ask for the accounts you want in plain language and the agent calls Fetch and Extract for you.

### Can I scrape X (Twitter) on a schedule?

Yes. Create a scheduled Batch job with your X (Twitter) URLs and the same settings as a single request, and Zenrows runs it on its side: every few hours, once a day, at set times on chosen days, or once at a future date. Each run keeps its own results, a webhook can notify your app when a run finishes, and the schedule can be paused or resumed at any time.

### Which integrations work with the Twitter scraper?

Zenrows works with n8n, Make, Zapier, Clay, Dify, LangChain, LlamaIndex, CrewAI and Mastra, plus the Python, Node.js and Go SDKs and any HTTP client. Clay users enrich a list of handles with bio and follower counts; n8n users write daily snapshots to Google Sheets. See the integrations page for the full list.

### What format does the data come in?

JSON by default, with the structured record in parsed and the original page in html . You can also ask for the page as Markdown or plain text, which is what most LLM and RAG pipelines want.

All scrapers: https://www.zenrows.com/scrape. Start free: https://app.zenrows.com/register. Docs: https://docs.zenrows.com