How to Implement Proxies With Undetected ChromeDriver

Aitor Mato
Aitor Mato
September 13, 2024 · 4 min read
Table of contents

Do you find yourself getting blocked despite employing the power of Undetected ChromeDriver? While this ChromeDriver patch is optimized to mask Selenium's bot-like properties, websites can still block your IP address.

In this tutorial, you'll learn how to implement a proxy with Undetected ChromeDriver to overcome this obstacle, including basic proxy setup, proxy rotation, authentication, and more.

Let's go!

Configure the Proxy With Undetected ChromeDriver

To implement proxies in Undetected ChromeDriver, you'll add your proxy address to Chrome options using Selenium's add_argument() method. Then, you'll initialize an Undetected ChromeDriver instance with these settings.

Let's see it in practice.

We'll use proxies from the Free Proxy List for demonstration purposes and request https://httpbin.io/ip, a test website that returns your IP address. Note that free proxies are unreliable due to their short lifespan. So, those used in this tutorial may not work at the time of reading. 

Additionally, we'll opt for HTTPS proxies as they also work with the HTTP protocol. A standard proxy address follows this pattern:

Example 
<PROXY_PROTOCOL>://<PROXY_IP_ADDRESS>:<PROXY_PORT>

To get started, install Undetected ChromeDriver using the following command:

Terminal 
pip3 install undetected-chromedriver

Import the library and By class from Selenium to select elements on the page and view your result:

Example 
# pip3 install undetected-chromedriver
import undetected_chromedriver as uc
from selenium.webdriver.common.by import By

Define your proxy details, set Chrome options, and add your proxy to the Chrome options using the add_argument() method:

Example 
# ...

# define proxy details
proxy = "http://20.44.189.184:3129"

if __name__ == "__main__":
    # set Chrome options
    options = uc.ChromeOptions()

    # run Chrome in headless mode
    options.headless = True

    # add proxy to Chrome options
    options.add_argument(f"--proxy-server={proxy}")

Lastly, initialize an Undetected ChromeDriver Chrome instance with the configured Chrome options. Open the target website (https://httpbin.io/ip) and print its text content to see your result:

Example 
# ...

if __name__ == "__main__":
   
    # ...

    # create a Chrome instance
    driver = uc.Chrome(
        use_subprocess=False,
        options=options,
    )

    # visit the test URL to check your proxy IP
    driver.get("https://httpbin.io/ip")

    # select the body tag containing the current IP address
    ip_address = driver.find_element(By.TAG_NAME, "body").text

    # print your current IP
    print(ip_address)

Combine the snippets. Here's the complete code:

Example 
# pip3 install undetected-chromedriver
import undetected_chromedriver as uc
from selenium.webdriver.common.by import By

# define proxy details
proxy = "http://20.44.189.184:3129"

if __name__ == "__main__":
    # set Chrome options
    options = uc.ChromeOptions()

    # run Chrome in headless mode
    options.headless = True

    # add proxy to Chrome options
    options.add_argument(f"--proxy-server={proxy}")

    # create a Chrome instance
    driver = uc.Chrome(
        options=options,
        use_subprocess=False,
    )

    # # visit the test URL to check your proxy IP
    driver.get("https://httpbin.io/ip")

    # select the body tag containing the current IP address
    ip_address = driver.find_element(By.TAG_NAME, "body").text

    # print your current IP
    print(ip_address)

Run the above Python code, and you'll get your proxy's IP address:

Output 
{
  "origin": "20.44.189.184:44884"
}

Awesome! You just set your first proxy in Undetected ChromeDriver. What if you're using a paid proxy that requires authentication? In the next section, you'll see how to handle that.

Frustrated that your web scrapers are blocked once and again?
ZenRows API handles rotating proxies and headless browsers for you.
Try for FREE

Proxy Authentication for Selenium Undetected ChromeDriver

While free proxies are suitable for learning and testing, they have a short lifespan, are often unreliable, and don't work in real-world use cases. That's because they're public, and many people use them simultaneously.

On the other hand, premium web scraping proxies offer reliability and better performance. Commercial premium proxy providers often require authentication to ensure that only authorized users can use their services. This authentication process takes several forms, with most involving credentials, such as username and password.

An authenticated proxy usually takes the following format:

Example 
<PROXY_PROTOCOL>://<YOUR_USERNAME>:<YOUR_PASSWORD>@<PROXY_IP_ADDRESS>:<PROXY_PORT>

Since Selenium doesn't support basic authentication, you must integrate with other libraries to achieve this. This tutorial uses Selenium Wire, a Python library that extends Selenium's capabilities by providing access to browser APIs for modifying requests and responses.

To authenticate an Undetected ChromeDriver proxy using this library, configure your proxy settings in the seleniumwire_options attribute of the ChromeDriver. 

Let's see it in practice. 

Start by installing Selenium Wire using pip:

Terminal 
pip3 install blinker==1.7.0 selenium-wire

Import Undetected ChromeDriver from Selenium Wire:

Example 
# pip3 install selenium-wire
import seleniumwire.undetected_chromedriver as uc

Define your proxy settings, including the username and password. Use these settings to form your proxy address and add it to proxy options:

Example 
# ...

# define your proxy credentials
proxy_username = "<YOUR_USERNAME>"
proxy_password = "<YOUR_PASSWORD>"
proxy_host = "<PROXY_IP_ADDRESS>"
proxy_port = "<PROXY_PORT>"

# form the proxy address
proxy_address = f"http://{proxy_username}:{proxy_password}@{proxy_host}:{proxy_port}"

# add the proxy address to proxy options
proxy_options = {
    "proxy": {
        "http": proxy_address,
        "https": proxy_address,
    }
}

Initialize an Undetected ChromeDriver instance with the specified proxy settings using the seleniumwire_options argument. Open the target website and print its body element to view the proxy IP address:

Example 
# ...

if __name__ == "__main__":
    # set Chrome options
    options = uc.ChromeOptions()

    # run Chrome in headless mode
    options.headless = True

    # create a Chrome instance with the proxy options
    driver = uc.Chrome(
        seleniumwire_options=proxy_options,
        options = options,
        use_subprocess=False,
    )

    # # visit the test URL to check your proxy IP
    driver.get("https://httpbin.io/ip")

    # select the body tag containing the current IP address
    ip_address = driver.find_element(By.TAG_NAME, "body").text

    # print your current IP
    print(ip_address)

Combine the snippets, and you'll have the following complete code:

Example 
# pip3 install selenium-wire
import seleniumwire.undetected_chromedriver as uc
from selenium.webdriver.common.by import By

# define your proxy credentials
proxy_username = "<YOUR_USERNAME>"
proxy_password = "<YOUR_PASSWORD>"
proxy_host = "<PROXY_IP_ADDRESS>"
proxy_port = "<PROXY_PORT>"

# form the proxy address
proxy_address = f"http://{proxy_username}:{proxy_password}@{proxy_host}:{proxy_port}"

# add the proxy address to proxy options
proxy_options = {
    "proxy": {
        "http": proxy_address,
        "https": proxy_address,
    }
}

if __name__ == "__main__":
    # set Chrome options
    options = uc.ChromeOptions()
    
    # run Chrome in headless mode
    options.headless = True

    # create a Chrome instance with the proxy options
    driver = uc.Chrome(
        seleniumwire_options=proxy_options,
        options = options,
        use_subprocess=False,
    )

    # # visit the test URL to check your proxy IP
    driver.get("https://httpbin.io/ip")

    # select the body tag containing the current IP address
    ip_address = driver.find_element(By.TAG_NAME, "body").text

    # print your current IP
    print(ip_address)

That's it! Your scraper now uses an authenticated proxy.

Rotating Proxies With Undetected ChromeDriver

A single proxy doesn't guarantee consistent access because some websites use rate limiting to block IPs that request beyond a specific traffic limit. Additionally, requesting frequently with a single proxy might reduce its IP quality over time, making it vulnerable to an IP ban.

To avoid that, you need to rotate your proxies. 

Proxy rotation is a technique that cycles IPs per request from a pool of several IP addresses, distributing traffic across many machines and avoiding potential IP bans. It's handy for mimicking different users, especially when scraping multiple pages.

You'll see how to rotate proxies in Undetected ChromeDriver in the next section.

Rotating Free IPs With Selenium and Undetected ChromeDriver

To rotate free proxies with Selenium and Undetected ChromeDriver, we'll grab the proxies from the Free Proxy List and use them to create a custom list in our script. Again, keep in mind that free proxies are unreliable. You'll need to get quality IPs for a real-life project.

First, add Python's built-in random and itertools modules to your imports and create your custom proxy list. You'll use the random library to shuffle the proxy list and use itertools to create a generator to rotate its items:

Example 
# ...

import itertools
import random

# define a proxy list
proxy_pool = [
    "http://23.247.136.245:80",
    "http://133.130.107.58:80",
    "http://123.30.154.171:7777",
]

Define a function to rotate the proxies. This function accepts a proxy_list argument and shuffles the list before cycling through it as a generator, ensuring the scraper uses a random proxy per request:

Example 
# ...

# define a function to rotate the proxies
def rotate_proxy(proxy_list: list):
    # shuffle the proxy list
    random.shuffle(proxy_list)

    # return a generator that cycles through the proxy list
    return itertools.cycle(proxy_list)

Create a generator instance from the rotator function and add it as the proxy option:

Example 
# ...

# create a generator instance
proxy_generator = rotate_proxy(proxy_pool)

if __name__ == "__main__":

    # ...

    # add the rotated proxy to Chrome options
    options.add_argument(f"--proxy-server={next(proxy_generator)}")

Update the previous single proxy code with these changes. Your complete code should look like this:

Example 
# pip3 install undetected-chromedriver
import undetected_chromedriver as uc
from selenium.webdriver.common.by import By
import itertools
import random

# define a proxy list
proxy_pool = [
    "http://23.247.136.245:80",
    "http://133.130.107.58:80",
    "http://123.30.154.171:7777",
]

# define a function to rotate the proxies
def rotate_proxy(proxy_list: list):
    # shuffle the proxy list
    random.shuffle(proxy_list)

    # return a generator that cycles through the proxy list
    return itertools.cycle(proxy_list)

# create a generator instance
proxy_generator = rotate_proxy(proxy_pool)

if __name__ == "__main__":
    # set Chrome options
    options = uc.ChromeOptions()

    # add the rotated proxy to Chrome options
    options.add_argument(f"--proxy-server={next(proxy_generator)}")

    options.headless = True

    # create a Chrome instance
    driver = uc.Chrome(
        use_subprocess=False,
        options=options,
    )

    # visit the test URL to check your proxy IP
    driver.get("https://httpbin.io/ip")

    # select the body tag containing the current IP address
    ip_address = driver.find_element(By.TAG_NAME, "body").text

    # print your current IP
    print(ip_address)

Running the above code a few times prints a random IP per request, indicating your scraper now rotates proxies. Here's a sample output for four consecutive requests:

Output 
# request 1
{
    "origin": "123.30.154.171:53652"
}

# request 2
{
    "origin": "133.130.107.58:66535"
}


# request 3
{
    "origin": "23.247.136.245:6527"
}


# request 4
{
    "origin": "23.247.136.245:88265"
}

Bravo! You now know how to rotate IPs from a custom proxy list in Undetected ChromeDriver.

Premium Residential Proxies to Get Unblocked

As mentioned earlier, free proxies eventually get blocked due to low quality. 

The best option is to use premium residential proxies. These proxies use IPs assigned to daily network users by internet service providers (ISPs), making them highly reliable and less likely to get blocked.

There are many proxy providers out there. Choose a provider that offers advanced features like proxy auto-rotation and geo-targeting. More importantly, your proxy provider should have extra features for bypassing complex anti-bot protections, such as anti-CAPTCHA, anti-bot auto-bypass, and more. ZenRows offers all these features at a unified price.

With ZenRows, your proxies are less likely to get blocked. Even if they do, you can get through using its anti-bot bypass toolkit, which includes a built-in headless browser, request header management, anti-CAPTCHA, and more.

Let's quickly see how ZenRows' residential proxies work.

Sign up to open the Request Builder. Go to the Proxy Generator by clicking Residential Proxies. Copy your proxy credentials (username and password) and the proxy domain and proxy port. Then, paste them into your Undetected ChromeDriver scraper.

generate residential proxies with zenrows
Click to open the image in full screen

As a reminder, if using Undetected ChromeDriver, use Selenium Wire to add your proxy authentication credentials like so:

Example 
# pip3 install selenium-wire
import seleniumwire.undetected_chromedriver as uc

# define your proxy credentials
proxy_username = "<ZENROWS_PROXY_USERNAME>"
proxy_password = "<ZENROWS_PROXY_PASSWORD>"
proxy_host = "<PROXY_DOMAIN>"
proxy_port = "<PROXY_PORT>"

# form the proxy address
proxy_address = f"http://{proxy_username}:{proxy_password}@{proxy_host}:{proxy_port}"

# add the proxy address to proxy options
proxy_options = {
    "proxy": {
        "http": proxy_address,
        "https": proxy_address,
    }
}

if __name__ == "__main__":
    # set Chrome options
    options = uc.ChromeOptions()
    
    # run Chrome in headless mode
    options.headless = True

    # create a Chrome instance with the proxy options
    driver = uc.Chrome(
        seleniumwire_options=proxy_options,
        options = options,
        use_subprocess=False,
    )

    # visit the test URL to check IP
    driver.get("https://httpbin.io/ip")

    # select the body tag containing the current IP address
    ip_address = driver.find_element(By.TAG_NAME, "body").text

    # print your current IP
    print(ip_address)

Here are the outputs for three requests:

Output 
# request 1
{
    "origin": "194.230.160.129:23859"
}
# request 2
{
    "origin": "77.89.83.231:61500"
}
# request 3
{
  "origin": "83.28.15.130:48140"
}

Congratulations 🎉! Your scraper now uses ZenRows residential proxies to bypass potential IP bans and geo-restrictions. 

Still getting blocked by advanced anti-bot protections, such as CAPTCHAs and web application firewalls (WAFs)? Feel free to bypass them with the ZenRows scraping API.

Conclusion

You've learned different ways to set up a proxy with Selenium and Undetected ChromeDriver in Python, including basic proxy setup, authentication, and IP rotation. 

Above all, rotating quality residential IPs is essential for guaranteed success during scraping. ZenRows is one of the best residential proxy providers, with advanced scraping features to bypass any anti-bot protection at scale.

Try ZenRows's residential proxy solution today and avoid potential bans during scraping.

Share
Link copied!

Ready to get started?

Up to 1,000 URLs for free are waiting for you
Try ZenRows for Free