--- title: "Google Scholar scraper" description: "Scrape Google Scholar results as JSON: titles, authors, venues, years, citation counts, versions, PDF links and related searches. One API request from Python or MCP." canonical: https://www.zenrows.com/scrape/google-scholar updated: 2026-10-20 --- # Google Scholar scraper The Zenrows Google Scholar scraper turns any Scholar search URL into JSON: paper title, authors, publication, year, publisher, snippet, cited-by count, number of versions, Scholar ID and full-text PDF link for every result, plus related searches and the total result count. Fetch in Adaptive Stealth Mode opens the page and Extract returns the fields. Run it from an API call, the CLI or an AI agent. - Site: scholar.google.com - Returns: Papers with authors, venues, years, citation counts and PDF links ## Run it: one Scholar search in, papers out ```json Input { "url": "https://scholar.google.com/scholar?q=large+language+models", "mode": "auto", "extract": "auto" } ``` ```json Output { "search_query": { "terms": "large language models" }, "result_summary": { "total_results_approximate": 6650000, "response_time_seconds": 0.11 }, "articles": [ { "title": "A survey on evaluation of large language models", "authors": ["Y Chang", "X Wang", "J Wang", "Y Wu", "L Yang"], "publication_name": "ACM transactions on …", "publication_year": 2024, "publisher_or_host": "dl.acm.org", "cited_by_count": 6954, "versions_count": 11, "scholar_id": "o93zfHYlUTIJ", "article_url": "https://dl.acm.org/doi/abs/10.1145/3641289", "fulltext_format": "PDF", "fulltext_url": "https://dl.acm.org/doi/pdf/10.1145/3641289" }, { "title": "Large language models: A survey", "authors": ["S Minaee", "T Mikolov", "N Nikzad", "M Chenaghlu"], "publication_name": "arXiv preprint arXiv …", "publication_year": 2024, "publisher_or_host": "arxiv.org", "cited_by_count": 3613, "versions_count": 7, "scholar_id": "msVvatpT9iUJ", "article_url": "https://arxiv.org/abs/2402.06196", "fulltext_format": "PDF", "fulltext_url": "https://arxiv.org/pdf/2402.06196" } ] } ``` Request and output of a live call to a Google Scholar search on Oct 20, 2026, trimmed to two of the ten articles returned. A field is returned when the result shows it. ## What data can you scrape from Google Scholar? Every Scholar result is a paper record: who wrote it, where and when it appeared, how often it is cited and where to read it. Extract returns each one in `articles`, in ranking order, with the search context around it. | Field | What it holds | Example | |---|---|---| | `articles[].title` | Paper title | `A survey on evaluation of large language models` | | `articles[].authors[]` | Author names as Scholar abbreviates them | `["Y Chang", "X Wang", …]` | | `articles[].publication_name` | Journal, conference or preprint server | `ACM transactions on …` | | `articles[].publication_year` | Year of publication | `2024` | | `articles[].publisher_or_host` | Publisher or the site hosting the paper | `dl.acm.org` | | `articles[].snippet` | The text excerpt Scholar shows for the query | `… 3.1 Natural Language Processing Tasks …` | | `articles[].cited_by_count` | "Cited by" count | `6954` | | `articles[].versions_count` | Number of versions Scholar grouped together | `11` | | `articles[].scholar_id` | Scholar's ID for the result, stable between runs | `o93zfHYlUTIJ` | | `articles[].article_url` | Link to the paper's landing page | `https://dl.acm.org/doi/abs/10.1145/3641289` | | `articles[].fulltext_url` | Direct full-text link, when Scholar shows one | `https://dl.acm.org/doi/pdf/10.1145/3641289` | | `articles[].fulltext_format` | Format of that full text | `PDF`, `HTML` | | `articles[].all_versions_url` | Scholar link to every version of the paper | `/scholar?cluster=3625720365842685347…` | | `articles[].related_articles_url` | Scholar's "Related articles" link | `/scholar?q=related:o93zfHYlUTIJ…` | | `result_summary.total_results_approximate` | Scholar's "About N results" count | `6650000` | | `related_searches[]` | Suggested queries with their Scholar URLs | `{ "query_text": "prompt large language models" }` | ## How to scrape Google Scholar with Zenrows 1. **Create a free Zenrows account** and copy your API key from the dashboard. 2. **Build the Scholar URL.** `https://scholar.google.com/scholar?q=` plus your terms. Add `as_ylo=2023` to start from a year, `as_yhi` to end at one, and `start=10` for the second page of ten. 3. **Add `hl=en` for consistent labels.** Scholar's interface language follows the visitor, and an English interface keeps venue names and links uniform across runs. 4. **Send it with the default configuration**: Adaptive Stealth Mode to open the page and Extract to return the fields. The sample above ran without a country. 5. **Run topics and pages in Batch**, with an ID per query and page, when one search returns what you want. ```python Python # pip install requests import requests params = { "apikey": "YOUR_ZENROWS_API_KEY", "url": "https://scholar.google.com/scholar?q=large+language+models", "mode": "auto", # Adaptive Stealth Mode "extract": "auto", # structured JSON } response = requests.get("https://api.zenrows.com/v1/", params=params, timeout=180) response.raise_for_status() parsed = response.json()["parsed"] print(parsed["result_summary"]["total_results_approximate"], "results") for paper in parsed["articles"]: print(paper["publication_year"], paper["cited_by_count"], paper["title"]) ``` ```javascript Node.js // Node 18+ (built-in fetch) const params = new URLSearchParams({ apikey: "YOUR_ZENROWS_API_KEY", url: "https://scholar.google.com/scholar?q=large+language+models", mode: "auto", extract: "auto", }); const response = await fetch(`https://api.zenrows.com/v1/?${params}`, { signal: AbortSignal.timeout(180_000), }); if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`); const { parsed } = await response.json(); for (const a of parsed.articles) console.log(a.publication_year, a.cited_by_count, a.title); ``` ```bash cURL curl "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fscholar.google.com%2Fscholar%3Fq%3Dlarge%2Blanguage%2Bmodels&mode=auto&extract=auto" # Only titles, years, citation counts and PDF links curl "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fscholar.google.com%2Fscholar%3Fq%3Dlarge%2Blanguage%2Bmodels&mode=auto&extract=auto&extract_fields=articles.title%2Carticles.publication_year%2Carticles.cited_by_count%2Carticles.fulltext_url" ``` ```python Python SDK # pip install zenrows from zenrows import ZenRowsClient client = ZenRowsClient("YOUR_ZENROWS_API_KEY") response = client.get( "https://scholar.google.com/scholar?q=large+language+models&as_ylo=2025", params={"mode": "auto", "extract": "auto"}, ) for paper in response.json()["parsed"]["articles"]: if paper.get("fulltext_format") == "PDF": print(paper["title"], paper["fulltext_url"]) ``` ## Input The scraper reads one Scholar results URL per request. Terms, year range and page number are all URL parameters, so a literature sweep is a list of URLs sent with the settings below. | Parameter | Type | Default | Description | |---|---|---|---| | `url` | string | required | A Scholar results URL, such as `https://scholar.google.com/scholar?q=large+language+models`. Add `as_ylo`/`as_yhi` for years, `start` for pages, `cites=` for papers citing one result | | `mode` | string | `auto` | Adaptive Stealth Mode: starts with the lightest setup and adds rendering and protected-site handling only when Scholar needs them | | `extract` | string | `auto` | Returns `articles`, `related_searches`, `result_summary` and `search_query` as JSON in `parsed` | | `extract_fields` | string | all fields | Comma-separated paths to keep, such as `articles.title,articles.cited_by_count,articles.scholar_id` | | `proxy_country` | string | any | Two-letter country code. Rarely needed for Scholar; useful when a journal link or interface language depends on location | | `wait_for` | string | none | A CSS selector to wait for before the page is read | | `session_id` | integer | none | Keep the same session for up to 10 minutes while you page through one query | ```json Batch input { "type": "regular", "status": "closed", "zenrows_params": { "mode": "auto", "extract": "auto" }, "tasks": [ { "url": "https://scholar.google.com/scholar?q=large+language+models", "external_id": "llm-p1" }, { "url": "https://scholar.google.com/scholar?q=large+language+models&start=10", "external_id": "llm-p2" }, { "url": "https://scholar.google.com/scholar?q=retrieval+augmented+generation&as_ylo=2024", "external_id": "rag-2024" } ] } ``` ## Output The response holds `html`, the Scholar page as served, and `parsed`, the structured results. Scholar links inside the results (all versions, related articles, related searches) are relative paths; prefix them with `https://scholar.google.com` to request them. Trimmed from the live response: ```json Search results { "search_query": { "terms": "large language models" }, "result_summary": { "response_time_seconds": 0.11, "total_results_approximate": 6650000 }, "articles": [ { "title": "A comprehensive overview of large language models", "authors": ["H Naveed", "AU Khan", "S Qiu", "M Saqib", "S Anwar"], "publication_name": "ACM Transactions on …", "publication_year": 2025, "publisher_or_host": "dl.acm.org", "snippet": "… Large Language Models (LLMs) have recently demonstrated remarkable capabilities in \nnatural language processing tasks and beyond…", "cited_by_count": 3843, "versions_count": 10, "scholar_id": "GWi_IEWnRVoJ", "article_url": "https://dl.acm.org/doi/abs/10.1145/3744746", "fulltext_format": "PDF", "fulltext_url": "https://dl.acm.org/doi/pdf/10.1145/3744746", "all_versions_url": "/scholar?cluster=6504789152150546457&hl=en&as_sdt=0,33", "related_articles_url": "/scholar?q=related:GWi_IEWnRVoJ:scholar.google.com/&scioq=large+language+models&hl=en&as_sdt=0,33" } ], "related_searches": [ { "query_text": "emergent large language models", "search_url": "/scholar?hl=en&as_sdt=0,33&qsp=1&q=emergent+large+language+models&qst=ib" }, { "query_text": "prompt large language models", "search_url": "/scholar?hl=en&as_sdt=0,33&qsp=3&q=prompt+large+language+models&qst=ib" } ] } ``` Scholar truncates long venue names and author lists with `…`; `article_url` leads to the full record on the publisher's site. For a reading list, `pd.json_normalize(parsed["articles"])` gives one paper per row, ready for CSV, a reference manager import or a database. ## Scrape Google Scholar from your coding agent Claude Code, Cursor, Codex and any MCP client can search Google Scholar and read papers as JSON from a prompt. The agent runs the same Fetch and Extract request as the code above, so it can rank papers by citations, open the PDF links and summarize them in one session. Connect the Zenrows MCP server once, or install the CLI and let the agent call it in the terminal. ```bash Connect the MCP server (Claude Code) claude mcp add zenrows -e ZENROWS_API_KEY=YOUR_ZENROWS_API_KEY -- npx -y @zenrows/mcp ``` ```json MCP config (Cursor, Claude Desktop, Windsurf) { "mcpServers": { "zenrows": { "command": "npx", "args": ["-y", "@zenrows/mcp"], "env": { "ZENROWS_API_KEY": "YOUR_ZENROWS_API_KEY" } } } } ``` ```bash Or run the CLI npx -y @zenrows/cli init zenrows extract "https://scholar.google.com/scholar?q=large+language+models" ``` ```text Then ask your agent Use Zenrows to search Google Scholar for "retrieval augmented generation" from 2024 onward. Read the first three pages, list the ten most cited papers with year, venue and PDF link, and summarize the three newest. ``` ## Scrape Google Scholar at scale with Batch and Crawl **Batch** runs a literature sweep as one job. Each task is one query and page (`start=0`, `10`, `20`), the ID names both, and results come back tagged so you can merge pages and drop duplicates by `scholar_id`. Rerun it monthly to watch citation counts grow. **Crawl** takes over once you have the papers. Scholar points to publishers and repositories; start a crawl on a repository or lab publication list you found there, keep the links that lead to papers (on arXiv, `/abs/`), and collect the abstracts behind every result. ```bash Batch (cURL) curl -X POST "https://async.api.zenrows.com/v1/jobs" \ -H "X-API-Key: YOUR_ZENROWS_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "type": "regular", "status": "closed", "zenrows_params": { "mode": "auto", "extract": "auto" }, "tasks": [ { "url": "https://scholar.google.com/scholar?q=large+language+models", "external_id": "llm-p1" }, { "url": "https://scholar.google.com/scholar?q=large+language+models&start=10", "external_id": "llm-p2" } ] }' # Read the results when the job finishes curl "https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful" \ -H "X-API-Key: YOUR_ZENROWS_API_KEY" ``` ```bash Batch (CLI) # queries.jsonl: one {"url": "...", "external_id": "...", "extract": "auto"} per line zenrows batch create queries.jsonl --wait zenrows batch results --out scholar-papers.jsonl ``` ```bash Crawl (cURL) curl -X POST "https://api.zenrows.com/v1/crawls" \ -H "X-API-Key: YOUR_ZENROWS_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://arxiv.org/list/cs.CL/recent", "depth": 1, "max_pages": 100, "include_patterns": ["/abs/"] }' ``` **Scheduled jobs** run the same Batch job on a timetable, on Zenrows' side: every few hours, once a day, at set times on chosen days, or once at a future date. Each run keeps its own results, a webhook can tell your app when a run finishes, and you can pause or resume the schedule at any time, with no cron server to keep alive. ```bash Scheduled job (cURL) # Runs every day on Zenrows' side; each run keeps its own results curl -X POST "https://async.api.zenrows.com/v1/jobs" \ -H "X-API-Key: YOUR_ZENROWS_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "type": "scheduled", "status": "closed", "schedule": { "rate": { "every": 1, "unit": "day" } }, "zenrows_params": { "mode": "auto", "extract": "auto" }, "tasks": [ { "url": "https://scholar.google.com/scholar?q=large+language+models", "external_id": "google-scholar" } ] }' # Or at set times: "schedule": { "calendar": { "times_of_day": ["09:00"], # "cadence": { "weekly": { "days": ["mon", "thu"] } } }, "timezone": "Europe/Madrid" } ``` ## What teams do with Google Scholar data | Use case | Data they pull | Who runs it | |---|---|---| | Literature reviews | Titles, authors, years, venues and PDF links for every query in a review protocol | Researchers, PhD students and research librarians | | Citation tracking | Cited-by counts per paper, month over month | Labs, universities and research offices | | Research trend analysis | Result counts and top papers per topic and year range | Analysts, funders and science journalists | | Expert and partner discovery | The authors behind the most cited work on a topic | R&D, biotech and pharma teams | | Training and RAG corpora | Paper metadata plus full-text links to fetch | AI and data teams | | Competitive R&D intelligence | Papers by a company's researchers and where they publish | Corporate strategy and IP teams | ## Tips for scraping Google Scholar - **Page with `start`.** Scholar shows ten results per page; `start=10` is page two. Each page is its own URL, so put them all in one Batch job. - **Narrow by year in the URL.** `as_ylo=2024` keeps papers from 2024 on, `as_yhi=2022` stops at 2022. Year slices also help with broad topics, since each slice brings different papers forward. - **Follow "Cited by" with `cites=`.** The citing papers of a result are a Scholar results page of their own; request it like any search and read `articles` again. - **Deduplicate on `scholar_id`.** The same paper can rank for several queries; its Scholar ID stays the same, while titles can differ slightly between versions. - **Prefix relative links.** `all_versions_url`, `related_articles_url` and `related_searches[].search_url` start with `/scholar`; add `https://scholar.google.com` before requesting them. - **Use phrase quotes for exact terms.** `q=%22large+language+models%22` matches the exact phrase, which cuts noise on common words. ## Frequently asked questions ### Is it legal to scrape Google Scholar? Scholar's results pages, with titles, authors, venues and citation counts, are public and shown to anyone who searches, and collecting bibliographic metadata for research is common. Author names are personal data under laws such as the GDPR, so use them for the research purpose you collect them for. Check Google's terms and the publishers' terms for full texts, and ask your lawyer if you are unsure. ### Is there an official Google Scholar API? No. Google has never published an API for Scholar. Open alternatives such as Crossref, OpenAlex and Semantic Scholar offer APIs with their own coverage, but Scholar's ranking, cited-by counts and version grouping are only on the Scholar pages, which is why researchers use a scraper for them. ### What Google Scholar pages can I scrape? Search results ( scholar.google.com/scholar?q=… ), including year-filtered searches, later pages, "Cited by" lists ( cites= ), all-versions lists ( cluster= ) and related-article lists. All of them are Scholar results pages, built like the search itself. ### Can I get citation counts from Google Scholar? Yes. Every result carries cited_by_count , and versions_count for the number of grouped versions. Run the same queries on a schedule to track how citations grow. ### Can I download PDFs from Google Scholar results? Scholar links a full text when one is available, and the scraper returns it in fulltext_url with its format. Request that URL with Fetch to get the file or page, and respect the publisher's terms for full texts. ### How do I scrape Google Scholar with Python? Send a GET request to https://api.zenrows.com/v1/ with your API key, the Scholar URL, Adaptive Stealth Mode and Extract turned on, as in the Python example above. response.json()["parsed"]["articles"] is a list of paper dictionaries. The zenrows package on PyPI wraps the same request. ### How many results can I scrape from Google Scholar? Each page returns ten results. Send every page and every year slice as its own URL in a Batch job, which runs them in parallel and keeps your ID on each result. One Batch job accepts more than 10,000 URLs, inline or as a CSV file. ### Can AI agents scrape Google Scholar with Zenrows? Yes. Connect the Zenrows MCP server to Claude, Cursor, Codex or any MCP client, or give your agent the Zenrows CLI. Ask for papers on a topic and the agent runs Fetch and Extract, reads the JSON and can open the full texts. ### Can I scrape Google Scholar on a schedule? Yes. Create a scheduled Batch job with your Google Scholar URLs and the same settings as a single request, and Zenrows runs it on its side: every few hours, once a day, at set times on chosen days, or once at a future date. Each run keeps its own results, a webhook can notify your app when a run finishes, and the schedule can be paused or resumed at any time. ### Which integrations work with the Google Scholar scraper? Zenrows works with n8n, Make, Zapier, Clay, Dify, LangChain, LlamaIndex, CrewAI and Mastra, plus the Python, Node.js and Go SDKs and any HTTP client. LlamaIndex and LangChain suit research assistants that read the papers they find. See the integrations page for the full list. ### What format does the data come in? JSON by default, with the papers in parsed and the original page in html . You can also ask for Markdown or plain text, or convert the JSON to CSV for a spreadsheet or reference manager. All scrapers: https://www.zenrows.com/scrape. Start free: https://app.zenrows.com/register. Docs: https://docs.zenrows.com