How to Bypass CAPTCHA With Selenium

Having trouble getting past CAPTCHA with Selenium? Maybe it keeps showing up, even after your script clicks through it. Don’t worry—we’ll walk you through two reliable methods to bypass CAPTCHA using Selenium.

Why Selenium Alone Isn't Enough to Bypass CAPTCHA?

Being explicitly designed for automation testing, Selenium lacks the essential mechanisms to bypass CAPTCHAs independently during scraping. It leaks bot-like parameters, such as the presence of a WebDriver, incomplete request headers, missing fingerprints, and more.

Websites often enhance CAPTCHA protection with background checks, such as browser fingerprinting, request header analysis, behavioral analysis, etc. These initial tests usually trigger CAPTCHAs when they detect automation. Selenium won't pass CAPTCHA by default unless you modify it to evade these detection mechanisms. 

The following methods are trusted solutions to bypass CAPTCHA with Selenium.

Method #1: Bypass CAPTCHA With SeleniumBase and Undetected ChromeDriver

Most protected websites block Selenium bots because its base version sends clear bot signals, often triggering a CAPTCHA. SeleniumBase is an open-source Python library used for automation testing and web scraping. It works with the Undetected ChromeDriver to patch the base Selenium loopholes and bypass anti-bot detection.

Let's see how it works by scraping this Cloudflare Challenge page, a website protected by the Cloudflare Turnstile CAPTCHA.

First, install SeleniumBase using pip:

pip3 install seleniumbase

Now, import the Driver class from SeleniumBase. Then, instantiate the browser in non-headless mode using the Undetected ChromeDriver helper and open the target site with a reconnection rule:

# pip3 install seleniumbase
from seleniumbase import Driver

# initialize driver in GUI mode with UC enabled
driver = Driver(uc=True, headless=False)

# set the target URL
url = "https://www.scrapingcourse.com/cloudflare-challenge"

# open URL using UC mode with 6 second reconnect time
driver.uc_open_with_reconnect(url, reconnect_time=6)

Note that the target website may trigger a CAPTCHA box. Since SeleniumBase can bypass some initial mechanisms, you can automate clicking the CAPTCHA box to access the destination page. However, that functionality only works with SeleniumBase in the graphical user interface mode (non-headless).

You can implement CAPTCHA clicking with the uc_gui_click_captcha method and screenshot the website to confirm if SeleniumBase with Undetected ChromeDriver bypassed the anti-bot:

# ...
# attempt to bypass CAPTCHA if present using UC mode's built-in method
driver.uc_gui_click_captcha()

# take a screenshot of the current page and save it
driver.save_screenshot("screenshot.png")

# close the browser and end the session
driver.quit()

Combine both snippets, and you'll get the following final code:

# pip3 install seleniumbase
from seleniumbase import Driver

# initialize driver in GUI mode with UC enabled
driver = Driver(uc=True, headless=False)

# set the target URL
url = "https://www.scrapingcourse.com/cloudflare-challenge"

# open URL using UC mode with 6 second reconnect time
driver.uc_open_with_reconnect(url, reconnect_time=6)

# attempt to bypass CAPTCHA if present using UC mode's built-in method
driver.uc_gui_click_captcha()

# take a screenshot of the current page and save it
driver.save_screenshot("screenshot.png")

# close the browser and end the session
driver.quit()

The above code opens the browser in GUI mode and simulates CAPTCHA clicking to access the protected page. See the result below:

Great! SeleniumBase with the Undetected ChromeDriver bypassed the CAPTCHA challenge.

However, CAPTCHA clicking is unavailable in headless mode, which is the ideal environment for web scraping. This limitation makes the setup impractical for large-scale scraping since running a GUI browser instance becomes expensive and unmanageable at scale. Additionally, anti-bots frequently update their security measures, making it challenging for open-source tools like SeleniumBase to keep up with evolving detection methods.

Considering these limitations, you need a sustainable and scalable approach to bypassing CAPTCHA while web scraping or crawling. Keep reading to learn about the tool that works all the time.

Method #2: ZenRows' Universal Scraper API to Bypass CAPTCHA

Extracting content from websites with advanced protection requires reliable tooling, especially if you're scraping multiple pages. 

The easiest and most efficient way to avoid any form of CAPTCHA is to use a web scraping solution like the ZenRows' Universal Scraper API. With a single API call in any programming language, it handles all the technicalities of avoiding anti-bot detection mechanisms under the hood. 

ZenRows fortifies your scraper with premium proxy rotation, advanced fingerprinting evasions, JavaScript rendering support, CAPTCHA and anti-bot auto-bypass, and more. It also has a headless browser feature to automate user interactions, making it a solid replacement for Selenium. 

Unlike the previous solution, which takes up extra bandwidth with a browser GUI instance, the ZenRows' Universal Scraper API only requires a lightweight HTTP client like Python's Requests.

Let's see how the ZenRows' Universal Scraper API works by scraping the previous protected page. 

First, sign up on ZenRows to open the Universal Scraper API Request Builder. Paste your target URL in the link box and activate Premium Proxies and JS Rendering.

Select your programming language (Python, in this case) and choose the API connection mode. Copy the generated Python code and paste it into your scraper.

Here's what the generated Python code looks like:

# pip3 install requests
import requests

url = "https://www.scrapingcourse.com/cloudflare-challenge"
apikey = "<YOUR_ZENROWS_API_KEY>"
params = {
    "url": url,
    "apikey": apikey,
    "js_render": "true",
    "premium_proxy": "true",
}
response = requests.get("https://api.zenrows.com/v1/", params=params)
print(response.text)

The code outputs the protect page's HTML with text proving you bypassed the CAPTCHA challenge:

<html lang="en">
<head>
    <!-- ... -->
    <title>Cloudflare challenge - ScrapingCourse.com</title>
    <!-- ... -->
</head>
<body>
    <!-- ... -->
    <h2>
        You bypassed the Cloudflare challenge! :D
    </h2>
    <!-- other content omitted for brevity -->
</body>
</html>

Congratulations! 🎉 You just bypassed CAPTCHA using the ZenRows' Universal Scraper API. You can now scale your scraper to any extent without getting blocked.

Conclusion

You've learned to bypass CAPTCHAs using Selenium with open-source and paid solutions. The open-source option can become expensive as you scale up. The open-source option can become costly as you scale up and is vulnerable to anti-bot mechanisms eventually catching on. While the paid solution requires some initial financial commitment, it's the most reliable for long-term scalable data extraction.

ZenRows provides the easiest and most efficient way to avoid anti-bot detection and bypass any CAPTCHA challenge. It's user-friendly and superfast, allowing you to balance performance with scalability. Try ZenRows for free today!