--- title: "Google News scraper" description: "Scrape Google News results as JSON: headlines, publishers, authors, publish times, article links and thumbnails. One API request from Python, Node.js, CLI or MCP." canonical: https://www.zenrows.com/scrape/google-news updated: 2026-10-20 --- # Google News scraper The Zenrows Google News scraper turns a Google News search or topic URL into structured data: headline, publisher, author, publish time, position, article link, thumbnail and publisher icon for every story. You send the URL, Fetch in Adaptive Stealth Mode opens the page, and Extract returns the fields. Run it from an API call, the CLI or an AI agent. - Site: news.google.com - Returns: Headlines, publishers, authors, publish times and article links ## Run it: one Google News search in, JSON out ```json Input { "url": "https://news.google.com/search?q=artificial%20intelligence&hl=en-US&gl=US&ceid=US:en", "mode": "auto", "extract": "auto", "proxy_country": "us" } ``` ```json Output { "search_query": { "text": "artificial intelligence" }, "articles": [ { "position": 1, "title": "The AI doom scenario you've never heard of may be the hardest to stop", "source_name": "CBS News", "author": "Shaun Raviv", "published_date": "10 hours ago", "url": "https://news.google.com/read/CBMieEFVX3lxTE5mMmdEVkNzd21XbDlBaTE1YTdQNTU0NjBjYXgw…?hl=en-US&gl=US&ceid=US:en", "thumbnail_url": "https://news.google.com/api/attachments/CC8iK0NnNXRORXA2V1VsQmEzUk9hbVJJVFJDUkF4ajhCU2dLTWdZWlVaRHJJUWM=-w200-h112-p-df-rw", "source_icon_url": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https%3A%2F%2Fwww.cbsnews.com&client=NEWS_360&size=96&type=FAVICON&fallback_opts=TYPE%2CSIZE%2CURL" }, { "position": 2, "title": "Nvidia CEO urges German manufacturers to adopt AI, saying digital tech won't replace heavy industry", "source_name": "The Washington Post", "author": "David Mchugh", "published_date": "34 minutes ago", "url": "https://news.google.com/read/CBMi9gFBVV95cUxNTlJHWWVYa0tPWC02S3cxa1pEX1pNQ0duVUpwdExo…?hl=en-US&gl=US&ceid=US:en", "thumbnail_url": "https://news.google.com/api/attachments/CC8iL0NnNHhTMkZNYUU0NFRUUnZWbXB4VFJERUF4aW1CU2dLTWdtUkVaYW1FbXM5cFFJ=-w200-h112-p-df-rw", "source_icon_url": "https://encrypted-tbn3.gstatic.com/faviconV2?url=https%3A%2F%2Fwww.washingtonpost.com&client=NEWS_360&size=96&type=FAVICON&fallback_opts=TYPE%2CSIZE%2CURL" } ] } ``` Sample output for a Google News search page, built from a live request on Oct 20, 2026, trimmed to two of the stories the page listed. A field is returned when the page shows it. ## What data can you scrape from Google News? Google News lists stories as cards: a publisher, a headline, a time, often a byline and a thumbnail. The scraper returns each card as one record, in the order Google shows them, under the query or section you asked for. | Field | What it holds | Example | |---|---|---| | `search_query.text` | The search Google News answered | `artificial intelligence` | | `articles[]` | Every story card on the page, in page order | `{ "position": 1, "title": "The AI doom scenario…", … }` | | `articles[].position` | The card's place in the list | `1` | | `articles[].title` | The headline as Google News shows it | `EXCLUSIVE: How one hacker used AI agents and a few thousand dollars to break into scores of sites` | | `articles[].source_name` | The publisher | `Reuters` | | `articles[].author` | The byline, when the card shows one | `Raphael Satter` | | `articles[].published_date` | The time shown on the card, relative to when you scraped | `4 hours ago` | | `articles[].url` | Google News link to the story, which opens the publisher's article | `https://news.google.com/read/CBMixgFBVV95…` | | `articles[].thumbnail_url` | The story image Google News serves | `https://news.google.com/api/attachments/CC8iJ0Nn…-w200-h112-p-df-rw` | | `articles[].source_icon_url` | The publisher's favicon, with its domain in the `url` parameter | `…faviconV2?url=https%3A%2F%2Fwww.reuters.com…` | On a search for a busy topic, one page carries dozens of stories: the sample search for "artificial intelligence" listed 58 distinct articles from outlets including Reuters, CBS News, The Washington Post, Yahoo Finance and Gizmodo. ## How to scrape Google News with Zenrows 1. **Create a free Zenrows account** and copy your API key from the dashboard. 2. **Build the Google News URL.** A search is `https://news.google.com/search?q=` plus your query. Keep the `hl`, `gl` and `ceid` parameters: they set the edition, and the same query returns different outlets in `US:en` and `GB:en`. 3. **Narrow the query on Google's side.** `when:1d` or `when:7d` inside `q` limits results to the last day or week, and `site:reuters.com` keeps one publisher. 4. **Send it with the default configuration**: Adaptive Stealth Mode to open the page, Extract to return the fields, and the country that matches the edition. Read `parsed` from the response. 5. **Schedule it.** Google News changes by the hour, so run your query list in Batch on a timer and keep only article URLs you have not seen before. ```python Python # pip install requests import json import requests params = { "apikey": "YOUR_ZENROWS_API_KEY", "url": "https://news.google.com/search?q=artificial%20intelligence&hl=en-US&gl=US&ceid=US:en", "mode": "auto", # Adaptive Stealth Mode "extract": "auto", # structured JSON "proxy_country": "us", # match the US English edition } response = requests.get("https://api.zenrows.com/v1/", params=params, timeout=180) response.raise_for_status() data = response.json()["parsed"] for article in data["articles"]: print(article["position"], article["source_name"], article["published_date"], article["title"]) print(json.dumps(data["articles"][0], indent=2)) ``` ```javascript Node.js // Node 18+ (built-in fetch) const params = new URLSearchParams({ apikey: "YOUR_ZENROWS_API_KEY", url: "https://news.google.com/search?q=artificial%20intelligence&hl=en-US&gl=US&ceid=US:en", mode: "auto", extract: "auto", proxy_country: "us", }); const response = await fetch(`https://api.zenrows.com/v1/?${params}`, { signal: AbortSignal.timeout(180_000), }); if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`); const { parsed } = await response.json(); for (const a of parsed.articles) console.log(a.source_name, a.title); ``` ```bash cURL curl "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fnews.google.com%2Fsearch%3Fq%3Dartificial%2520intelligence%26hl%3Den-US%26gl%3DUS%26ceid%3DUS%3Aen&mode=auto&extract=auto&proxy_country=us" # Only headline, publisher and time curl "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fnews.google.com%2Fsearch%3Fq%3Dartificial%2520intelligence%26hl%3Den-US%26gl%3DUS%26ceid%3DUS%3Aen&mode=auto&extract=auto&proxy_country=us&extract_fields=articles.title%2Carticles.source_name%2Carticles.published_date" ``` ```python Python SDK # pip install zenrows from zenrows import ZenRowsClient client = ZenRowsClient("YOUR_ZENROWS_API_KEY") response = client.get( "https://news.google.com/search?q=artificial%20intelligence%20when%3A1d&hl=en-US&gl=US&ceid=US:en", params={"mode": "auto", "extract": "auto", "proxy_country": "us"}, ) for article in response.json()["parsed"]["articles"]: print(article["source_name"], article.get("author"), article["title"]) ``` ## Input The scraper takes one Google News URL per request. The query, edition and time filter are part of the URL; the options below control how Zenrows opens it. In a Batch job they apply to every task. | Parameter | Type | Default | Description | |---|---|---|---| | `url` | string | required | A Google News page: search (`/search?q=…`), topic (`/topics/…`) or the headlines page, with `hl`, `gl` and `ceid` for the edition | | `mode` | string | `auto` | Adaptive Stealth Mode: starts with the lightest setup and adds rendering and protected-site handling only when the page needs them | | `extract` | string | `auto` | Returns the story cards as JSON in `parsed`, next to the raw `html` | | `extract_fields` | string | all fields | Comma-separated paths to keep, such as `articles.title,articles.source_name,articles.url` | | `proxy_country` | string | any | Two-letter country code. Pick the one that matches the `gl` edition in the URL | | `wait_for` | string | none | A CSS selector to wait for before the page is read | | `session_id` | integer | none | Keep the same session across several requests, up to 10 minutes | ```json Batch input { "type": "regular", "status": "closed", "zenrows_params": { "mode": "auto", "extract": "auto", "proxy_country": "us" }, "tasks": [ { "url": "https://news.google.com/search?q=artificial%20intelligence&hl=en-US&gl=US&ceid=US:en", "external_id": "ai-us" }, { "url": "https://news.google.com/search?q=nvidia%20when%3A1d&hl=en-US&gl=US&ceid=US:en", "external_id": "nvidia-24h-us" }, { "url": "https://news.google.com/search?q=site%3Areuters.com%20semiconductors&hl=en-US&gl=US&ceid=US:en", "external_id": "reuters-semis-us" } ] } ``` ## Output Every response is JSON with two keys: `html`, the page as Google News served it, and `parsed`, the structured list of stories. The sample is trimmed to three cards of a real search page. ```json Search results { "search_query": { "text": "artificial intelligence" }, "articles": [ { "position": 1, "title": "The AI doom scenario you've never heard of may be the hardest to stop", "source_name": "CBS News", "author": "Shaun Raviv", "published_date": "10 hours ago", "url": "https://news.google.com/read/CBMieEFVX3lxTE5mMmdEVkNzd21XbDlBaTE1YTdQNTU0NjBjYXgw…?hl=en-US&gl=US&ceid=US:en", "thumbnail_url": "https://news.google.com/api/attachments/CC8iK0NnNXRORXA2V1VsQmEzUk9hbVJJVFJDUkF4ajhCU2dLTWdZWlVaRHJJUWM=-w200-h112-p-df-rw" }, { "position": 4, "title": "EXCLUSIVE: How one hacker used AI agents and a few thousand dollars to break into scores of sites", "source_name": "Reuters", "author": "Raphael Satter", "published_date": "4 hours ago", "url": "https://news.google.com/read/CBMixgFBVV95cUxQSUVKUWRvY0tQeUZscHhGcDBLYjI5VVZaYXRF…?hl=en-US&gl=US&ceid=US:en", "thumbnail_url": "https://news.google.com/api/attachments/CC8iJ0NnNHRSRmh2YWxob2JqTTBZVEpYVFJERUF4aW1CU2dLTWdPQjBRSQ=-w200-h112-p-df-rw" }, { "position": 5, "title": "Billionaire David Tepper's Appaloosa Management Has 31% of its portfolio Invested in These 2 Artificial Intelligence (AI) Stocks", "source_name": "Yahoo Finance", "author": null, "published_date": "4 hours ago", "url": "https://news.google.com/read/CBMirwFBVV95cUxNMk56NmR5dUZvRTVNTEdMLVJaTEs3dzBpcUZK…?hl=en-US&gl=US&ceid=US:en", "thumbnail_url": "https://news.google.com/api/attachments/CC8iL0NnNHpiM05EUkdGU1dFUlFSakZKVFJERUF4aW1CU2dLTWdrQmtaYUR0aWMxemdF=-w200-h112-p-df-rw" } ] } ``` Publish times are relative, as Google shows them. Turn them into timestamps by subtracting from the time of your request, and store that timestamp with the record. Need a spreadsheet? Load `articles` into pandas with `pd.json_normalize(...)` and write CSV. Need the page for an LLM? Ask for Markdown instead of JSON. ## Scrape Google News from your coding agent Claude Code, Cursor, Codex and any MCP client can pull the latest Google News coverage of a topic from a prompt. The agent runs the same Fetch and Extract request a developer would, so it reads headlines, publishers and times as data instead of guessing from memory. Connect the Zenrows MCP server once, or install the CLI and let the agent call it in the terminal. ```bash Connect the MCP server (Claude Code) claude mcp add zenrows -e ZENROWS_API_KEY=YOUR_ZENROWS_API_KEY -- npx -y @zenrows/mcp ``` ```json MCP config (Cursor, Claude Desktop, Windsurf) { "mcpServers": { "zenrows": { "command": "npx", "args": ["-y", "@zenrows/mcp"], "env": { "ZENROWS_API_KEY": "YOUR_ZENROWS_API_KEY" } } } } ``` ```bash Or run the CLI npx -y @zenrows/cli init zenrows extract "https://news.google.com/search?q=artificial%20intelligence&hl=en-US&gl=US&ceid=US:en" --proxy-country us ``` ```text Then ask your agent Use Zenrows to get the Google News US results for "artificial intelligence when:1d". Group the stories by publisher, list each headline with its time and author, and flag any story that three or more outlets are covering. ``` ## Scrape Google News at scale with Batch and Crawl **Batch** runs a list of Google News URLs as one job: one per query, edition and time filter, each with your own ID. A media monitoring setup runs it every hour for a list of brand names and competitors, then keeps the article URLs it has not stored before. **Crawl** follows the links on a start page and returns every page it reaches on news.google.com. Start it on a topic page and keep links that contain `/read/` to collect the story links in that section, or keep `/topics/` to map the sub-topics under it. Send the pages it returns to Batch with Extract for the structured cards. ```bash Batch (cURL) curl -X POST "https://async.api.zenrows.com/v1/jobs" \ -H "X-API-Key: YOUR_ZENROWS_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "type": "regular", "status": "closed", "zenrows_params": { "mode": "auto", "extract": "auto", "proxy_country": "us" }, "tasks": [ { "url": "https://news.google.com/search?q=artificial%20intelligence&hl=en-US&gl=US&ceid=US:en", "external_id": "ai-us" }, { "url": "https://news.google.com/search?q=nvidia%20when%3A1d&hl=en-US&gl=US&ceid=US:en", "external_id": "nvidia-24h-us" } ] }' # Read the results when the job finishes curl "https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful" \ -H "X-API-Key: YOUR_ZENROWS_API_KEY" ``` ```bash Batch (CLI) # queries.jsonl: one {"url": "https://news.google.com/search?q=...", "external_id": "...", "extract": "auto"} per line zenrows batch create queries.jsonl --proxy-country us --wait zenrows batch results --out google-news.jsonl ``` ```bash Crawl (cURL) curl -X POST "https://api.zenrows.com/v1/crawls" \ -H "X-API-Key: YOUR_ZENROWS_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://news.google.com/search?q=artificial%20intelligence&hl=en-US&gl=US&ceid=US:en", "depth": 1, "max_pages": 100, "include_patterns": ["/read/"] }' ``` **Scheduled jobs** run the same Batch job on a timetable, on Zenrows' side: every few hours, once a day, at set times on chosen days, or once at a future date. Each run keeps its own results, a webhook can tell your app when a run finishes, and you can pause or resume the schedule at any time, with no cron server to keep alive. ```bash Scheduled job (cURL) # Runs every day on Zenrows' side; each run keeps its own results curl -X POST "https://async.api.zenrows.com/v1/jobs" \ -H "X-API-Key: YOUR_ZENROWS_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "type": "scheduled", "status": "closed", "schedule": { "rate": { "every": 1, "unit": "day" } }, "zenrows_params": { "mode": "auto", "extract": "auto", "proxy_country": "us" }, "tasks": [ { "url": "https://news.google.com/search?q=artificial%20intelligence&hl=en-US&gl=US&ceid=US:en", "external_id": "google-news" } ] }' # Or at set times: "schedule": { "calendar": { "times_of_day": ["09:00"], # "cadence": { "weekly": { "days": ["mon", "thu"] } } }, "timezone": "Europe/Madrid" } ``` ## What teams do with Google News data | Use case | Data they pull | Who runs it | |---|---|---| | Media monitoring | Every headline that names a brand, with publisher, author and time, collected hourly | PR and communications teams | | Share of voice | Story counts per company and per outlet for the same set of queries over time | Marketing and analyst relations teams | | News sentiment for trading and research | Headlines and publishers for a ticker or company name, fed to a sentiment model | Quant, fintech and research teams | | Competitor tracking | Launches, funding and hiring news on rival companies, by date and outlet | Strategy and product marketing teams | | News feeds for AI agents and RAG | Fresh headlines and article links on a topic, with sources the model can cite | AI and data teams | | Journalist and outlet research | Which reporters and publishers cover a beat, from the bylines on each card | PR agencies and media relations teams | ## Tips for scraping Google News - **Keep the edition parameters.** `hl=en-US&gl=US&ceid=US:en` is the US English edition; `hl=en-GB&gl=GB&ceid=GB:en` is the UK one. Set the request country to match, or you get one country's edition with another country's local stories. - **Use Google's own filters in the query.** `when:1h`, `when:1d` and `when:7d` limit by recency, `site:` keeps one publisher, and quotes force an exact phrase. All of them go inside `q`, URL-encoded. - **Convert relative times right away.** "34 minutes ago" only means something next to the time you scraped. Compute the timestamp when the response arrives and store it with the article. - **Deduplicate on the article link.** The same story shows up in several searches and on topic pages. The `/read/` token in `url` is stable for one article, so use it as your key. - **Get the publisher's domain from the icon.** `source_icon_url` carries the publisher's site in its `url` parameter, which is handy when `source_name` is a brand name such as "Yahoo Finance". - **Run often, request less.** For monitoring, a narrow query with `when:1d` every hour catches more than a broad query once a day, because Google News reorders stories as they age. ## Frequently asked questions ### Is it legal to scrape Google News? Collecting public listings from Google News, such as headlines, publishers and times shown to every visitor, is common practice for media monitoring and research. Bylines are names of people, and personal data falls under laws such as the GDPR and the CCPA, so keep only what you have a legitimate reason to use. The articles themselves belong to their publishers. Check Google's and each publisher's terms for your use case and ask your lawyer if you are unsure. ### Is there an official Google News API? No. Google does not offer an API for Google News. It does publish RSS feeds ( news.google.com/rss/search?q=… ) with a headline, link, source and date per item, which suit simple alerts. The page itself carries more: authors, thumbnails, publisher icons and the order Google ranks stories in. ### What Google News pages can I scrape? Search results ( /search?q=… ), topic pages ( /topics/… ) and the headlines page, in any edition. Each lists stories in the same card layout: publisher, headline, time, byline and image. ### Can I scrape the full text of the articles? Google News links to the publisher's site, and the article text lives there. Follow url to the publisher's page and request it with Fetch, as Markdown if an LLM will read it. Check the publisher's terms before storing article text. ### How do I scrape Google News for a specific country or language? Use that edition's hl , gl and ceid values in the URL and request from the same country. To compare coverage across markets, send one URL per edition in a single Batch job. ### How do I scrape Google News with Python? Send a GET request to https://api.zenrows.com/v1/ with your API key, the Google News URL, Adaptive Stealth Mode and Extract, as in the Python example above. parsed["articles"] is a list of dictionaries, one per story. The zenrows package on PyPI wraps the same request. ### Can AI agents scrape Google News with Zenrows? Yes. Connect the Zenrows MCP server to Claude, Cursor, Codex or any MCP client, or give your agent the Zenrows CLI. Ask for the latest coverage of a topic and the agent calls Fetch and Extract, then works from the returned headlines and links. ### Can I scrape Google News on a schedule? Yes. Create a scheduled Batch job with your Google News URLs and the same settings as a single request, and Zenrows runs it on its side: every few hours, once a day, at set times on chosen days, or once at a future date. Each run keeps its own results, a webhook can notify your app when a run finishes, and the schedule can be paused or resumed at any time. ### Which integrations work with the Google News scraper? Zenrows works with n8n, Make, Zapier, Clay, Dify, LangChain, LlamaIndex, CrewAI and Mastra, plus the Python, Node.js and Go SDKs and any HTTP client. A scheduled workflow can post new headlines to Slack or append them to a sheet. See the integrations page for the full list. ### What format does the data come in? JSON by default, with the stories in parsed and the original page in html . You can also ask for the page as Markdown or plain text, which keeps each headline next to its link for an LLM to read. All scrapers: https://www.zenrows.com/scrape. Start free: https://app.zenrows.com/register. Docs: https://docs.zenrows.com