Web Scraping Blog

Find out how to bypass Cloudflare with Scrapy. Bypass Cloudflare's systems and get the data you need with the right middleware.

Learn why the User Agent in Axios is getting your scraper blocked, how to change it and the steps to randomize it at scale.

Learn how to use Selenium with NodeJS for in this step-by-step tutorial. You'll master browser automation for multiple purposes.

Learn the ins and outs of Puppeteer Stealth evasions and discover how to give your scraper an extra anti-bot boost.

Learn how to set up AIOHTTP proxies: add a proxy to your script, build a proxy rotator, and discover how to use premium proxies to avoid blocks and bans.

Learn how to use curl_cffi for web scraping, from the basics to more advanced techniques such as concurrent requests and session handling.

Learn how to bypass CAPTCHA with Selenium in Ruby by using Undetected ChromeDriver or a web scraping API.

Learn how to efficiently parse HTML with Ruby and Nokogiri, from extracting data all the way to CSV export.

Learn how to take advantage of Golang's scraping power without writing a Go program and use Ferret for web scraping.

Learn how to add a proxy to MechanicalSoup. Build a proxy rotator and discover how to use premium proxies to avoid all blocks and bans.

Learn how to use Playwright with Go to scrape the web and successfully bypass all blocks and bans.

Learn how to avoid jsoup 403 forbidden error with custom User Agent, rotating proxies, and a web scraping API.

Should you choose Python or PHP for web scraping? In this article, we’ll compare the two across factors such as popularity, speed, data analysis capabilities, and more.

Learn how to scrape the web with nodriver: extract HTML, deal with dynamic content, and avoid all blocks and bans.

Learn how to use a single proxy, rotate proxies from a pool, or take advantage of premium proxies while web scraping with OkHttp.

Learn to automate web scraping using Playwright in C# with this detailed guide, covering installation, setup, and data extraction.

Should you use Axios, Got, or Fetch API for your project? This article compares the three tools based on factors such as performance, popularity, and browser support.

Learn how to bypass Akamai when web scraping with Playwright with three methods: stealth plugin, web scraping API, and premium proxies.