--- title: "Google Search scraper" description: "Scrape Google results as JSON: organic results, AI Overview, People Also Ask, related searches and shopping products. One API request from Python, Node.js or MCP." canonical: https://www.zenrows.com/scrape/google-search updated: 2026-10-20 --- # Google Search scraper The Zenrows Google Search scraper turns any Google results URL into JSON: organic titles, snippets, source names and URLs, the AI Overview summary, People Also Ask questions, related searches, and shopping products with price, rating and seller. Fetch in Adaptive Stealth Mode opens the page and Extract returns the fields. Run it from an API call, the CLI or an AI agent. - Site: google.com - Returns: Organic results, AI Overview, People Also Ask, related searches and shopping products ## Run it: one Google query in, JSON out ```json Input { "url": "https://www.google.com/search?q=best+running+shoes&hl=en&gl=us", "mode": "auto", "extract": "auto", "proxy_country": "us" } ``` ```json Output { "search_query": { "text": "best running shoes" }, "ai_overview": { "summary": "The best running shoes depend on your main use case, feet type, and distance: RunRepeat +2", "key_features": null }, "organic_results": [ { "title": "The Best Running Shoes I Tested in 2025", "source_name": "Runner's World", "snippet": "Dec 30, 2025 · Here are my top four trainers of 2025. Nike Vomero Premium: Asics Metaspeed Edge Tokyo: My Top Pick for Racing…", "url": "/goto?url=CAESmQEB6zswFZb4fb1JZykZSUR…", "price": null, "availability": null }, { "title": "Best Road Running Shoes for Men of 2026", "source_name": "Treeline Review", "snippet": "Aug 13, 2026 · We like the HOKA Mach 6, Salomon Aero Glide 4, and On Cloudeclipse for speed workouts and intervals…", "url": "/goto?url=CAESgQEB6zswFZuIRp5Q6iqOMJ2W…", "price": null, "availability": null } ], "people_also_ask": [ { "question": "Is Hoka or Asics better?" }, { "question": "How often should running shoes be replaced?" } ], "related_searches": [ { "query": "Best running shoes for beginners" }, { "query": "Best running shoes for road" } ], "shopping_products": [ { "title": "Brooks Hyperion Elite 5", "seller_name": "Brooks Running", "current_price": "$199.95", "original_price": "$275", "rating": 4.4, "review_count": "(215)", "badge": null, "availability_nearby": null } ] } ``` Request and output of a live call to a Google results page on Oct 20, 2026, trimmed for length. A field is returned when the page shows it. ## What data can you scrape from Google Search? A Google results page is several result types stacked on one URL. Extract returns each block as its own group, so the organic list, the AI Overview and the shopping carousel never mix. These are the fields teams read most, with their paths in the JSON. | Field | What it holds | Example | |---|---|---| | `search_query.text` | The query Google ran | `best running shoes` | | `organic_results[].title` | Title of each organic result, in ranking order | `The Best Running Shoes I Tested in 2025` | | `organic_results[].source_name` | Publisher or site name shown above the result | `Runner's World` | | `organic_results[].snippet` | The description text, with the date when Google shows one | `Dec 30, 2025 · Here are my top four…` | | `organic_results[].url` | The result link as it appears on the page | `/goto?url=CAESmQEB6zsw…` | | `organic_results[].price` | Price shown on a product result | `null` on editorial results | | `ai_overview.summary` | The text of Google's AI Overview, with its cited sources | `The best running shoes depend on…` | | `ai_overview.key_features` | Bullet points inside the AI Overview, when present | `null` | | `people_also_ask[].question` | Questions in the People Also Ask box | `How often should running shoes be replaced?` | | `related_searches[].query` | Related queries at the bottom of the page | `Best running shoes for beginners` | | `shopping_products[].title` | Product name in the shopping carousel | `Brooks Hyperion Elite 5` | | `shopping_products[].seller_name` | The store selling that offer | `Brooks Running` | | `shopping_products[].current_price` | Price as displayed, with currency symbol | `$199.95` | | `shopping_products[].original_price` | The struck-through price on a discounted offer | `$275` | | `shopping_products[].rating` | Product star rating | `4.4` | | `shopping_products[].review_count` | Review count as Google prints it | `(2.5K)` | | `shopping_products[].badge` | Labels such as a sale or popularity tag | `null` | ## How to scrape Google Search with Zenrows 1. **Create a free Zenrows account** and copy your API key from the dashboard. 2. **Build the results URL.** Start from `https://www.google.com/search?q=` and add your query with `+` between words. Add `hl=en` for the interface language and `gl=us` for the market whose rankings you want. 3. **Request from the same country as `gl`.** Google ranks by the visitor's location, so the country of the request and the `gl` value should agree, or the results drift between the two. 4. **Send it with the default configuration**: Adaptive Stealth Mode to open the page and Extract to return the fields. Read `parsed` and keep only the groups you need. 5. **Run your keyword list in Batch** once one query returns what you want, with the keyword as the ID on each task. ```python Python # pip install requests import json import requests params = { "apikey": "YOUR_ZENROWS_API_KEY", "url": "https://www.google.com/search?q=best+running+shoes&hl=en&gl=us", "mode": "auto", # Adaptive Stealth Mode "extract": "auto", # structured JSON "proxy_country": "us", # rankings as a US searcher sees them } response = requests.get("https://api.zenrows.com/v1/", params=params, timeout=180) response.raise_for_status() data = response.json()["parsed"] for rank, result in enumerate(data["organic_results"], start=1): print(rank, result["source_name"], result["title"]) print(json.dumps(data.get("people_also_ask"), indent=2)) ``` ```javascript Node.js // Node 18+ (built-in fetch) const params = new URLSearchParams({ apikey: "YOUR_ZENROWS_API_KEY", url: "https://www.google.com/search?q=best+running+shoes&hl=en&gl=us", mode: "auto", extract: "auto", proxy_country: "us", }); const response = await fetch(`https://api.zenrows.com/v1/?${params}`, { signal: AbortSignal.timeout(180_000), }); if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`); const { parsed } = await response.json(); parsed.organic_results.forEach((r, i) => console.log(i + 1, r.source_name, r.title)); console.log(parsed.ai_overview?.summary); ``` ```bash cURL curl "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fwww.google.com%2Fsearch%3Fq%3Dbest%2Brunning%2Bshoes%26hl%3Den%26gl%3Dus&mode=auto&extract=auto&proxy_country=us" # Only the organic titles, sources and People Also Ask curl "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fwww.google.com%2Fsearch%3Fq%3Dbest%2Brunning%2Bshoes%26hl%3Den%26gl%3Dus&mode=auto&extract=auto&proxy_country=us&extract_fields=organic_results.title%2Corganic_results.source_name%2Cpeople_also_ask" ``` ```python Python SDK # pip install zenrows from zenrows import ZenRowsClient client = ZenRowsClient("YOUR_ZENROWS_API_KEY") response = client.get( "https://www.google.com/search?q=best+running+shoes&hl=en&gl=us", params={"mode": "auto", "extract": "auto", "proxy_country": "us"}, ) parsed = response.json()["parsed"] for product in parsed.get("shopping_products", []): print(product["title"], product["seller_name"], product["current_price"]) ``` ## Input The scraper takes one Google results URL per request. Everything that shapes the results (query, language, market, page, time range) lives in the URL itself, so a keyword list becomes a list of URLs with the same settings below. | Parameter | Type | Default | Description | |---|---|---|---| | `url` | string | required | A Google results URL, such as `https://www.google.com/search?q=best+running+shoes&hl=en&gl=us`. Add `start=10` for page two, `tbs=qdr:w` for the past week | | `mode` | string | `auto` | Adaptive Stealth Mode: starts with the lightest setup and adds rendering and protected-site handling only when Google needs them | | `extract` | string | `auto` | Returns organic results, AI Overview, People Also Ask, related searches and shopping products as JSON in `parsed` | | `extract_fields` | string | all fields | Comma-separated paths to keep, such as `organic_results.title,organic_results.url,related_searches` | | `proxy_country` | string | any | Two-letter country code. Match it to the `gl` value in the URL so rankings reflect that market | | `wait_for` | string | none | A CSS selector to wait for before the page is read | | `session_id` | integer | none | Keep the same session for up to 10 minutes, useful when you page through one query | ```json Batch input { "type": "regular", "status": "closed", "zenrows_params": { "mode": "auto", "extract": "auto", "proxy_country": "us" }, "tasks": [ { "url": "https://www.google.com/search?q=best+running+shoes&hl=en&gl=us", "external_id": "best-running-shoes" }, { "url": "https://www.google.com/search?q=best+trail+running+shoes&hl=en&gl=us", "external_id": "best-trail-running-shoes" }, { "url": "https://www.google.com/search?q=best+running+shoes&hl=en&gl=us&start=10", "external_id": "best-running-shoes-p2" } ] } ``` ## Output Each response carries `html`, the results page as Google served it, and `parsed`, the structured record. The groups in `parsed` follow what Google put on that page: a commercial query adds `shopping_products`, a question query usually adds `ai_overview` and `people_also_ask`. Below is the full shape of the live response, trimmed to a few items per list. ```json Results page { "search_query": { "text": "best running shoes" }, "ai_overview": { "key_features": null, "summary": "The best running shoes depend on your main use case, feet type, and distance: RunRepeat +2" }, "organic_results": [ { "title": "The Best Running Shoes I Tested in 2025", "source_name": "Runner's World", "snippet": "Dec 30, 2025 · Here are my top four trainers of 2025. Nike Vomero Premium: Asics Metaspeed Edge Tokyo: My Top Pick for Racing…", "url": "/goto?url=CAESmQEB6zswFZb4fb1JZykZSUR…", "price": null, "availability": null }, { "title": "What are some good running shoes. : r/runninglifestyle", "source_name": "Reddit · r/runninglifestyle", "snippet": "I'm looking for good running shoes for training the one I got for training hurt my calf when running for long distances…", "url": "/goto?url=CAESlwEB6zswFYMxt0p1Z1w9ok2v…", "price": null, "availability": null }, { "title": "7 Best Road Running Shoes in 2026", "source_name": "RunRepeat", "snippet": "Sep 12, 2026 · ASICS Novablast 6 emerged as the best daily trainer among road running shoes…", "url": "/goto?url=CAESbwHrOzAV0lK0ZtkjjCNU4vAS…", "price": null, "availability": null } ], "people_also_ask": [ { "question": "Is Hoka or Asics better?" }, { "question": "How often should running shoes be replaced?" }, { "question": "What are the top 5 brands for running shoes?" } ], "related_searches": [ { "query": "Best running shoes for beginners" }, { "query": "Best running shoes for asphalt and concrete" }, { "query": "Best running shoes for beginners with bad knees" } ], "shopping_products": [ { "title": "Nike Men's Pegasus Premium", "seller_name": "nike.com", "current_price": "$187.97", "original_price": "$220", "rating": 4.3, "review_count": "(2.5K)", "badge": null, "availability_nearby": null }, { "title": "Asics Magic Speed 5", "seller_name": "ASICS", "current_price": "$180.00", "original_price": null, "rating": 4.8, "review_count": "(1K)", "badge": null, "availability_nearby": null } ] } ``` The array order is the order on the page, so an item's index plus one is its rank inside that block. Prices and review counts arrive as Google prints them (`$187.97`, `(2.5K)`); strip the symbols when you load them into a numeric column. Need the page as text for an LLM? Ask for Markdown instead of JSON. ## Scrape Google Search from your coding agent Claude Code, Cursor, Codex and any MCP client can run Google searches and read the results as JSON. The agent sends the same Fetch and Extract request shown above, so it reads the organic list, the AI Overview and the People Also Ask box as fields instead of guessing from page text. Connect the Zenrows MCP server once, or install the CLI and let the agent call it from the terminal. ```bash Connect the MCP server (Claude Code) claude mcp add zenrows -e ZENROWS_API_KEY=YOUR_ZENROWS_API_KEY -- npx -y @zenrows/mcp ``` ```json MCP config (Cursor, Claude Desktop, Windsurf) { "mcpServers": { "zenrows": { "command": "npx", "args": ["-y", "@zenrows/mcp"], "env": { "ZENROWS_API_KEY": "YOUR_ZENROWS_API_KEY" } } } } ``` ```bash Or run the CLI npx -y @zenrows/cli init zenrows extract "https://www.google.com/search?q=best+running+shoes&hl=en&gl=us" --proxy-country us ``` ```text Then ask your agent Use Zenrows to search Google in the US for "best running shoes" and "best trail running shoes". For each query, list the top 10 organic results (rank, site, title), the People Also Ask questions, and which sites appear in both lists. ``` ## Scrape Google Search at scale with Batch and Crawl **Batch** runs a keyword list as one job. Each query is one URL, the keyword is its `external_id`, and every result comes back tagged with it, which is how daily rank tracking across thousands of keywords and several countries stays joinable. Add a second URL with `start=10` per keyword when you track beyond the first page. **Crawl** fits the other side of a search project: the pages that rank. Collect the result URLs from your Batch output, then crawl a competitor's site from its home page and keep the article paths of the section you compete with (on Runner's World, gear articles live under `/gear/a`) to read every page they built for those queries. ```bash Batch (cURL) curl -X POST "https://async.api.zenrows.com/v1/jobs" \ -H "X-API-Key: YOUR_ZENROWS_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "type": "regular", "status": "closed", "zenrows_params": { "mode": "auto", "extract": "auto", "proxy_country": "us" }, "tasks": [ { "url": "https://www.google.com/search?q=best+running+shoes&hl=en&gl=us", "external_id": "best-running-shoes" }, { "url": "https://www.google.com/search?q=best+trail+running+shoes&hl=en&gl=us", "external_id": "best-trail-running-shoes" } ] }' # Read the results when the job finishes curl "https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful" \ -H "X-API-Key: YOUR_ZENROWS_API_KEY" ``` ```bash Batch (CLI) # keywords.jsonl: one {"url": "...", "external_id": "...", "extract": "auto"} per line zenrows batch create keywords.jsonl --proxy-country us --wait zenrows batch results --out serps.jsonl ``` ```bash Crawl (cURL) curl -X POST "https://api.zenrows.com/v1/crawls" \ -H "X-API-Key: YOUR_ZENROWS_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://www.runnersworld.com/gear/", "depth": 2, "max_pages": 200, "include_patterns": ["/gear/a"] }' ``` **Scheduled jobs** run the same Batch job on a timetable, on Zenrows' side: every few hours, once a day, at set times on chosen days, or once at a future date. Each run keeps its own results, a webhook can tell your app when a run finishes, and you can pause or resume the schedule at any time, with no cron server to keep alive. ```bash Scheduled job (cURL) # Runs every day on Zenrows' side; each run keeps its own results curl -X POST "https://async.api.zenrows.com/v1/jobs" \ -H "X-API-Key: YOUR_ZENROWS_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "type": "scheduled", "status": "closed", "schedule": { "rate": { "every": 1, "unit": "day" } }, "zenrows_params": { "mode": "auto", "extract": "auto", "proxy_country": "us" }, "tasks": [ { "url": "https://www.google.com/search?q=best+running+shoes&hl=en&gl=us", "external_id": "google-search" } ] }' # Or at set times: "schedule": { "calendar": { "times_of_day": ["09:00"], # "cadence": { "weekly": { "days": ["mon", "thu"] } } }, "timezone": "Europe/Madrid" } ``` ## What teams do with Google Search data | Use case | Data they pull | Who runs it | |---|---|---| | Rank tracking | Position, title and source of every organic result per keyword, country and day | SEO teams and agencies | | AI Overview monitoring | Whether an AI Overview appears, its summary text and the sources it cites | Content, brand and SEO teams | | Content research | People Also Ask questions and related searches for a topic cluster | Content strategists and writers | | Shopping visibility | Products, sellers and prices in the shopping carousel for commercial queries | Ecommerce and retail media teams | | Brand and reputation checks | What ranks for the brand name, review sites, forums and news in the top results | PR and brand teams | | Grounding AI agents and RAG | Fresh results for a question, before the agent opens the pages it needs | AI and data teams | ## Tips for scraping Google Search - **Pin the market twice.** Put `gl=` and `hl=` in the URL and send the request from the same country. A US query fetched from Germany returns German rankings with English labels. - **Page with `start`.** Google shows about ten organic results per page; `start=10` is page two and `start=20` page three. Request each page as its own URL in the same Batch job. - **Filter by date in the URL.** `tbs=qdr:d`, `qdr:w`, `qdr:m` and `qdr:y` limit results to the past day, week, month and year, which is the quickest way to watch news and fresh content for a query. - **Treat the index as the rank.** `organic_results` comes back in page order, so the position is the array index plus one. Store it with the date and country, since rankings move daily. - **Expect a different layout per intent.** "best running shoes" returns a shopping carousel and an AI Overview; a navigational query may return neither. Read groups with `.get()` so a missing block does not break the run. - **Keep the raw `html` for audits.** When a ranking looks odd, the original page in the same response shows exactly what Google served. - **Use the vertical scrapers for depth.** Shopping tab results, Maps places and Scholar papers have their own pages: see the [Google Shopping scraper](/scrape/google-shopping), the [Google Maps scraper](/scrape/google-maps) and the [Google Scholar scraper](/scrape/google-scholar). ## Frequently asked questions ### Is it legal to scrape Google Search? Search results are public pages shown to anyone who types a query, and collecting rankings, titles and snippets is standard practice in SEO and market research. Some results can contain personal data, which laws such as the GDPR and the CCPA protect, so keep only what you have a legitimate reason to use. Google's terms restrict automated access, so review them for your use case and ask your lawyer if you are unsure. ### Is there an official Google Search API? Google's Custom Search JSON API is closed to new customers. It searches a Programmable Search Engine rather than the google.com page a person sees, so it has no AI Overview, People Also Ask or shopping carousel. Teams that need the real results page use a scraper. ### What Google Search pages can I scrape? Any google.com/search results URL: web results, results filtered by date, later pages with start , and country sites such as google.co.uk or google.de. The Shopping tab, Images tab, Maps and Scholar have dedicated scrapers with their own fields. ### Can I get results for a specific country or language? Yes. Set gl and hl in the URL and choose the matching country for the request. Zenrows then opens the page from that location, so rankings, the AI Overview and shopping prices match what a searcher in that market sees. ### Does the scraper return Google's AI Overview? When Google shows an AI Overview for the query, Extract returns it in ai_overview with the summary text and its key points. Track the same queries every day to see when an overview appears, changes or cites your site. ### How do I scrape Google Search with Python? Send a GET request to https://api.zenrows.com/v1/ with your API key, the Google results URL, Adaptive Stealth Mode and Extract turned on, as in the Python example above. response.json()["parsed"] holds the results as a dictionary. The zenrows package on PyPI wraps the same request. ### How many keywords can I track? Send one query per request, or a full keyword list as a Batch job that runs the URLs in parallel and keeps your keyword ID on each result. One Batch job accepts more than 10,000 URLs, inline or as a CSV file. ### Can AI agents scrape Google Search with Zenrows? Yes. Connect the Zenrows MCP server to Claude, Cursor, Codex or any MCP client, or give your agent the Zenrows CLI. The agent searches Google, reads the results as JSON and opens the pages it needs, all through Fetch and Extract. ### Can I scrape Google Search on a schedule? Yes. Create a scheduled Batch job with your Google Search URLs and the same settings as a single request, and Zenrows runs it on its side: every few hours, once a day, at set times on chosen days, or once at a future date. Each run keeps its own results, a webhook can notify your app when a run finishes, and the schedule can be paused or resumed at any time. ### Which integrations work with the Google Search scraper? Zenrows works with n8n, Make, Zapier, Clay, Dify, LangChain, LlamaIndex, CrewAI and Mastra, plus the Python, Node.js and Go SDKs and any HTTP client. A daily rank report can run in n8n or Make and land in Google Sheets or a warehouse. See the integrations page for the full list. ### What format does the data come in? JSON by default, with the structured results in parsed and the original page in html . You can also ask for the results page as Markdown or plain text, which suits LLM prompts and RAG pipelines. All scrapers: https://www.zenrows.com/scrape. Start free: https://app.zenrows.com/register. Docs: https://docs.zenrows.com