Browser automation tools like Playwright and Puppeteer can't independently escape detection through browser fingerprinting. However, pairing them with the fingerprint-suite package can increase your chances of evading the detection mechanism.
We'll show you how the fingerprint-suite package works and how to use it to bypass browser fingerprinting while scraping with Playwright in Node.js.
What Is fingerprint-suite?
fingerprint-suite is a combination of tools for generating and injecting browser fingerprints into automated browser instances. And it works specifically with Puppeteer and Playwright.
By default, Puppeteer and Playwright leak bot-like fingerprints that make them detectable to anti-bot systems. These include the HeadlessChrome flag in the user agent string, the presence of navigator.webdriver, missing browser plugins, and other telltale signs.
The fingerprint-suite library replaces these bot-like fingerprints with genuine browser-like ones, increasing your chances of evading anti-bot detection during scraping. The suite features the following individual libraries:
- fingerprint-injector: Injects browser fingerprints directly into a browser instance.
- fingerprint-generator: Generates realistic browser fingerprints.
- header-generator: Generates configurable HTTP request headers.
- generative-bayesian-network: Also generates realistic browser fingerprints.
The fingerprint-injector library, however, combines the functionalities of the three other tools. It creates and injects realistic browser fingerprints and HTTP headers into browser instances.
In the next section, you'll find out how to use this suite, focusing on the fingerprint-injector library.
Bypass Browser Fingerprinting With fingerprint-suite During Scraping
In this section, you'll inject browser fingerprints into Playwright using fingerprint-suite. You'll then test the injection by requesting bot.sannysoft, a browser fingerprinting test site.
Before we begin, let's see what the standard Playwright looks like without fingerprint modification using the following code:
// npm install playwright
// npx playwright install
const { chromium } = require('playwright');
(async () => {
// launch a browser instance in headless mode
const browser = await chromium.launch();
const context = await browser.newContext();
const page = await context.newPage();
// visit the target page
await page.goto('https://bot.sannysoft.com/', { waitUntil: 'networkidle' });
// screenshot the page
await page.screenshot({
path: 'sannysoft-screenshot.png',
});
await browser.close();
})();
As shown below, the plain Playwright test fails major fingerprint tests, including User Agent, WebDriver, Chrome runtime, Permission, Plugins, and WebGL Renderer:
Let's improve this result with the fingerprint-injector library from fingerprint-suite.
As mentioned earlier, we'll focus on fingerprint-injector since it combines the features of the other libraries in the suite.
Install the library with npm:
npm install fingerprint-injector
You're now ready to begin!
Step 1: Inject the Fingerprints into a Browser Instance
First, import the Playwright and fingerprint-injector into your script. Launch a new browser instance and inject the browser fingerprints into a browser context. The following setup uses a mobile-based Chrome fingerprint on an iOS device:
// npm install playwright fingerprint-injector
// npx playwright install
const { chromium } = require('playwright');
const { newInjectedContext } = require('fingerprint-injector');
(async () => {
// launch a browser instance in headless mode
const browser = await chromium.launch();
// inject browser fingerprints into the browser instance
const context = await newInjectedContext(browser, {
fingerprintOptions: {
devices: ['mobile'],
browsers: ['chrome'],
operatingSystems: ['ios'],
},
newContextOptions: {
permissions: ['clipboard-read'],
},
});
await browser.close();
})();
Let's test the setup in the next step.
Step 2: Test Fingerprint Behavior
Now, update the previous code to send a request to the test website using the configured context. Then take the test screenshot:
// ...
(async () => {
// ...
// create a new page from the context
const page = await context.newPage();
// visit the target page
await page.goto('https://bot.sannysoft.com/');
// take a screenshot
await page.screenshot({ path: 'screenshot.png' });
await browser.close();
})();
Here's the complete code:
// npm install playwright fingerprint-injector
// npx playwright install
const { chromium } = require('playwright');
const { newInjectedContext } = require('fingerprint-injector');
(async () => {
// launch a browser instance in headless mode
const browser = await chromium.launch();
// inject browser fingerprints into the browser instance
const context = await newInjectedContext(browser, {
fingerprintOptions: {
devices: ['mobile'],
browsers: ['chrome'],
operatingSystems: ['ios'],
},
newContextOptions: {
permissions: ['clipboard-read'],
},
});
// create a new page from the context
const page = await context.newPage();
// visit the target page
await page.goto('https://bot.sannysoft.com/');
// take a screenshot
await page.screenshot({ path: 'screenshot.png' });
await browser.close();
})();
The above code patches the User Agent, WebDriver, Permissions, and WebGL Renderer fingerprints, as shown in the screenshot below. In addition to having a mobile (portrait) layout, the User Agent header now reflects iPhone OS. The WebGL Vendor and WebGL Renderer also show Apple Inc. and Apple GPU, respectively.
Nice! The fingerprint-injector library patched Playwright's browser instance with realistic fingerprints.
However, the patching is still incomplete. An insight into the tool's limitations will help you understand why this happened.
Limitations of fingerprint-suite
Although fingerprint-suite spoofs real browser fingerprints, it still doesn't capture all the essential leakages. The test result shows that the library still fails Plugins and Chrome runtime tests. No matter how small these leaks are, anti-bot measures can spot them quickly and block your scraping requests.
Another limitation of the tool is that it covers only a fraction of the detection mechanisms used in real-world applications. Anti-bots typically combine browser fingerprinting with other anti-scraping techniques, such as IP address analysis, JavaScript challenge, CAPTCHAs, behavioral tracking, TLS fingerprinting, etc.
The consequence of these limitations is that fingerprint-suite via the fingerprint-injector library can't handle advanced anti-bot protections. Even if your scraper evades browser fingerprinting by any chance, it won't pass other bot tests.
For instance, Playwright with fingerprint-injector gets blocked on a protected website like the Anti-bot Challenge page. Try it out with the current scraper:
// npm install playwright fingerprint-injector
// npx playwright install
const { chromium } = require('playwright');
const { newInjectedContext } = require('fingerprint-injector');
(async () => {
// launch a browser instance in headless mode
const browser = await chromium.launch();
// inject browser fingerprints into the browser instance
const context = await newInjectedContext(browser, {
fingerprintOptions: {
devices: ['mobile'],
browsers: ['chrome'],
operatingSystems: ['ios'],
},
newContextOptions: {
permissions: ['clipboard-read'],
},
});
// create a new page from the context
const page = await context.newPage();
// visit the target page
await page.goto('https://www.scrapingcourse.com/antibot-challenge');
// take a screenshot
await page.screenshot({ path: 'screenshot.png' });
await browser.close();
})();
The above script gets blocked as shown:
The next section shows you the best solution to bypass this anti-bot protection and scrape without getting blocked.
The Best Scraping Techniques to Avoid Detection
It's clear that fingerprint-suite is ineffective against advanced anti-bot measures. You might keep tweaking your scraper to bypass blocks by adding proxies or manual evasion patches. Unfortunately, this approach isn't sustainable because it can't keep up with the frequently updated anti-bot security measures.
The best way to scrape without getting blocked is to use a web scraping solution like the ZenRows Universal Scraper API.
With a single API call, ZenRows provides the following benefits:
- It helps you bypass the most advanced anti-bot systems and scrape any website on a large scale without limitations.
- It handles all evasion technicalities behind the scenes so that you can focus on core business logic.
- It saves time and resources required to solve scraping blocks manually.
- ZenRows lets you scrape localized content using globally distributed residential IPs with geo-targeting.
- It has lightweight, scalable headless browser features that can replace the memory-intensive Playwright browser instance.
Let's quickly see how the ZenRows Universal Scraper API works by scraping the Antibot Challenge page that blocked you previously.
Sign up and go to the Request Builder. Then, paste the target URL in the link box and activate Premium Proxies and JS Rendering.
Select Node.js as your preferred programming language and choose the API connection mode. Copy the generated code and paste it into your scraper.
The generated JavaScript looks like this:
// npm install axios
const axios = require('axios');
const url = 'https://www.scrapingcourse.com/antibot-challenge';
const apikey = '<YOUR_ZENROWS_API_KEY>';
axios({
url: 'https://api.zenrows.com/v1/',
method: 'GET',
params: {
url: url,
apikey: apikey,
js_render: 'true',
premium_proxy: 'true',
},
})
.then((response) => console.log(response.data))
.catch((error) => console.log(error));
The above request bypasses the anti-bot challenge and outputs the full-page HTML of the protected site. See the result below:
<html lang="en">
<head>
<!-- ... -->
<title>Antibot Challenge - ScrapingCourse.com</title>
<!-- ... -->
</head>
<body>
<!-- ... -->
<h2>
You bypassed the Antibot challenge! :D
</h2>
<!-- other content omitted for brevity -->
</body>
</html>
Congratulations! 🎉 You just used ZenRows to bypass an anti-bot measure. You can now scrape any website at scale, regardless of its anti-bot security level.
Conclusion
You've learned how to generate and inject real browser fingerprints into Playwright using the fingerprint-injector library from fingerprint-suite in Node.js. While the library patches your scraper to a reasonable extent, it still leaks bot-like fingerprints. This limits its ability to evade advanced protections.
To bypass any anti-bot measures reliably with minimal technicalities, we recommend using ZenRows. It's your one-stop solution for scraping at scale without limitations.