Web Scraping Blog

Scrapy in Python: Web Scraping Tutorial 2025

Learn how to use Scrapy in Python for web scraping and extract all the data you're interested in. Follow this step-by-step tutorial.

May 16, 2024 · 11 min read

Node Unblocker for Web Scraping: Step-by-Step Tutorial [2025]

Learn how to set up and configure Node Unblocker for web scraping to bypass IP blocks and rate limits. Follow this step-by-step tutorial.

Yuvraj Chandra

February 19, 2025 · 6 min read

How to Use Playwright Stealth for Scraping

Implement Playwright Stealth to web scrape without getting blocked and get access to the data you want. Follow this step-by-step tutorial.

July 27, 2023 · 5 min read

7 Best Scrapy Alternatives in 2025

Have a look at the best Scrapy alternatives, ordered by the overall top and by language. For both beginners and more advanced web scraping developers.

July 26, 2023 · 8 min read

How to Solve Scrapy 403 Forbidden Error

Learn why Scrapy Error 403 is being returned and how to solve this Forbidden error when web scraping. Get access to the data you want.

Sergio Nonide

July 25, 2023 · 7 min read

How to Use a Proxy with Pyppeteer in 2025

Discover the secret to using proxies with Pyppeteer in 2025! Learn the easy steps to leverage proxies for web scraping, automation, and staying anonymous online.

July 21, 2023 · 1 min read

Handle the User Agent in NodeJS

Learn how to set and rotate User Agents in NodeJS to successfully web scrape without getting blocked. You'll also find a handy tool to scale up.

Rubén del Campo

January 29, 2025 · 4 min read

How to Build a Web Crawler in Python

Learn about web crawling and how to build a Python web crawler through step-by-step examples. Also, dive into more advanced and actionable concepts.

Sergio Nonide

March 3, 2025 · 10 min read

CloudProxy Tutorial to Scrape Websites in 2025

Learn how to use CloudProxy to use proxies for bypassing anti-bot measures. Also, find a more reliable CloudProxy alternative to scrape any website.

July 12, 2023 · 5 min read

CloudUnflare for Real IP Address Reconnaissance

Discover how you can use CloudUnflare to bypass Cloudflare and find a more effective alternative for your scraping project.

July 10, 2023 · 3 min read

cURL User Agent: How to Set or Change It in 2025

Learn how to set, change, and randomize your cURL User Agent to bypass anti-bot measures and scrape any data you want.

Yuvraj Chandra

January 26, 2025 · 4 min read

10 Web Scraping Challenges You Should Know

Discover the most common web scraping challenges you're likely to encounter and how to resolve them quickly and effectively.

July 4, 2023 · 6 min read

How to Use a HttpClient Proxy in C# (2025)

Learn how to use a proxy with HttpClient to avoid IP blocks and scrape in C# quickly and effectively.

July 3, 2023 · 8 min read

2 Easy Ways to Bypass "Please Verify You Are a Human"

Discover the most efficient method to bypass the "Please verify you are human" challenge by PerimeterX and resume your scraping efforts.

Yuvraj Chandra

September 23, 2024 · 3 min read

How to Bypass Cloudflare in PHP (2 Methods)

Learn two effective methods to bypass Cloudflare in PHP to scrape any website and get the data you need.

Ander Rodriguez

September 19, 2024 · 4 min read

WhatWaf for Scraping Websites (2025)

Find out how you can use Whatwaf to bypass firewalls when web scraping and a better alternative for your next scraping project.

June 22, 2023 · 3 min read

Change the Selenium User Agent: Steps & Best Practices

Learn how to customize and rotate the Selenium User Agent when web scraping and find a solution to make the process easier.

Sergio Nonide

January 27, 2025 · 7 min read

Rust Web Scraping in 2025

Get started with web scraping in Rust following this step-by-step tutorial! Learn how to scrape a site and how to tackle the main challenges.

May 15, 2024 · 8 min read