Are you web scraping or testing a website with Playwright and want to take screenshots?
You're in the right place. In this article, you'll learn the three main methods of capturing screenshots with Playwright in Python:
- Generating a screenshot of the visible part of the screen.
- Capturing a full-page screenshot.
- Creating a screenshot of a specific site element.
Let's go!
How to Take a Screenshot With Playwright
Playwright is an open-source framework built on Node.js. It helps automate web browsing tasks and is known for its headless browser mode, which gives the tool a significant speed advantage. One of Playwright's key functionalities is screenshot-taking. Automated screenshotting has many commercial use cases, most notably in testing.
Depending on the part of the page you need to screenshot while scraping with Playwright, you will use a different method. In this section, you'll learn three ways of screenshotting different parts of a product page using ScrapingCourse.com as a demo web page.
Option 1: Generate a Screenshot of the Visible Part of the Screen
Grabbing the visible part of the screen is the easiest way to take a screenshot in Playwright. It only captures the section above the fold (the part of the page that appears immediately after loading).
Below, you can see an example of a screenshot made with this method:
Now, let's learn how to capture this type of screenshot.
First, start a synchronous Playwright chain and launch a new page instance:
# import the required library
from playwright.sync_api import sync_playwright
# initialize Playwright in synchronous mode
with sync_playwright() as p:
# launch the browser
browser = p.chromium.launch()
# create a new page instance
page = browser.new_page()
Visit the target website and initiate the screenshot method from the page instance. Then, close the browser:
# initialize Playwright in synchronous mode
with sync_playwright() as p:
# ...
# navigate to the target web page
page.goto('https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/')
# take a screenshot
page.screenshot(path='visible-part-of-the-page.png')
# close the browser
browser.close()
Merge everything. Your final code should look like this:
# import the required library
from playwright.sync_api import sync_playwright
# initialize Playwright in synchronous mode
with sync_playwright() as p:
# launch the browser
browser = p.chromium.launch()
# create a new page instance
page = browser.new_page()
# navigate to the target web page
page.goto('https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/')
# take a screenshot
page.screenshot(path='visible-part-of-the-page.png')
# close the browser
browser.close()
Awesome! The code above will screenshot the visible part of the target page. You'll find the output in your project directory.
Next, let's increase your coverage to a full page.
Option 2: Capture a Full-page Screenshot
Playwright lets you capture an entire web page, including the parts that require scrolling. For example, here's a full-page screenshot of the target product page:
To get the same result, you need to add a full_page argument to the screenshot method and set its condition to True:
# import the required library
from playwright.sync_api import sync_playwright
# initialize Playwright in synchronous mode
with sync_playwright() as p:
# launch the browser
browser = p.chromium.launch()
# create a new page instance
page = browser.new_page()
# navigate to the target web page
page.goto('https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/')
# take a full-page screenshot
page.screenshot(path='scrapingcourse-page-full-screenshot.png', full_page=True)
# close the browser
browser.close()
Yes, it's this easy! Now you know how to screenshot an entire page with Playwright.
Option 3: Create a Screenshot of a Specific Element
If you only want to screenshot a selected part of a page, Playwright also lets you capture its specific element. To do that, you'll have to point your web scraper to the target element.
For demonstration, let's grab a screenshot of the product summary container. It will look like this:
First, start Playwright in synchronous mode and launch a page instance:
# import the required library
from playwright.sync_api import sync_playwright
# initialize Playwright in synchronous mode
with sync_playwright() as p:
# launch the browser
browser = p.chromium.launch()
# create a new page instance
page = browser.new_page()
Next, open the web page and obtain the target element using its class attribute (.entry-summary). Then, screenshot it into the specified path and close the browser:
with sync_playwright() as p:
# ...
# navigate to the target web page
page.goto('https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/')
# obtain the target element
element = page.query_selector('.entry-summary')
# grab a screenshot of the target element
element.screenshot(path='scrapingcourse-specific_element_screenshot.png')
# close the browser
browser.close()
Here's the complete code:
# import the required library
from playwright.sync_api import sync_playwright
# initialize Playwright in synchronous mode
with sync_playwright() as p:
# launch the browser
browser = p.chromium.launch()
# create a new page instance
page = browser.new_page()
# navigate to the target web page
page.goto('https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/')
# obtain the target element
element = page.query_selector('.entry-summary')
# grab a screenshot of the target element
element.screenshot(path = 'scrapingcourse-specific_element_screenshot.png')
# close the browser
browser.close()
The code will return the expected screenshot. Congratulations!
Keep in mind that some websites implement anti-bot systems to block scrapers. How can you bypass them and grab the screenshot you want?
Avoid Getting Blocked While Taking Screenshots
Anti-bot measures may prevent you from taking screenshots while web scraping with Playwright.
For instance, the screenshot methods detailed in this guide won't work for protected websites such as G2.
Try replacing the target website with aG2 URL, as shown in the code below:
# import the required library
from playwright.sync_api import sync_playwright
# initialize Playwright in synchronous mode
with sync_playwright() as p:
# launch the browser
browser = p.chromium.launch()
# create a new page instance
page = browser.new_page()
# navigate to the target web page
page.goto('https://www.g2.com/products/asana/reviews')
# take a full-page screenshot
page.screenshot(path='g2-page-screenshot.png', full_page=True)
# close the browser
browser.close()
The code got blocked by Cloudflare Turnstile, and instead of the desired screenshot, it will return an image like this:
The best way to avoid this block is to use a web scraping API like ZenRows. It's an all-in-one web scraping solution that helps you fix your request headers, auto-rotate premium proxies, and bypass any anti-bot system.
Let's rerun the blocked screenshot request above using the ZenRows API instead of Playwright.
Sign up to open the Request Builder. Paste the target URL in the link box, set the Boost mode to JS Rendering, and activate Premium Proxies.
Select Python as your language choice. Next, copy the code generated and paste it into your scraper script.
You'll use the Requests library to send HTTP requests. Install it using pip:
pip install requests
The next step is modifying the generated code to take a screenshot with ZenRows.
First, import the Requests library and add a screenshot option to the request parameter:
# import the required library
import requests
# specify the query parameters with a screenshot option
params = {
'url': 'https://www.g2.com/products/asana/reviews',
'apikey': '<YOUR_ZENROWS_API_KEY>',
'js_render': 'true',
'premium_proxy': 'true',
'return_screenshot': 'true'
}
Next, send the request through the ZenRows API and set the response type to stream to read the generated image. Confirm the response, and write the generated screenshot into your project folder as a PNG:
# ...
# send the rquest and set the response type to stream
response = requests.get('https://api.zenrows.com/v1/', params=params, stream=True)
# check if the request was successful
if response.status_code == 200:
# save the response content as a screenshot
with open('g2-page-screenshot.png', 'wb') as f:
f.write(response.content)
print('Screenshot saved successfully.')
else:
print(f'Error: {response.status_code}')
Put the short snippets together, and your complete code should look like this:
# import the required library
import requests
# specify the query parameters with a screenshot option
params = {
'url': 'https://www.g2.com/products/asana/reviews',
'apikey': '<YOUR_ZENROWS_API_KEY>',
'js_render': 'true',
'premium_proxy': 'true',
'return_screenshot': 'true'
}
# send the rquest and set the response type to stream
response = requests.get('https://api.zenrows.com/v1/', params=params, stream=True)
# check if the request was successful
if response.status_code == 200:
# save the response content as a screenshot
with open('g2-page-screenshot.png', 'wb') as f:
f.write(response.content)
print('Screenshot saved successfully.')
else:
print(f'Error: {response.status_code}')
The code screenshots the entire protected page, including the parts that require scrolling.
Congratulations! You used ZenRows to access a protected page and captured a full-length screenshot without issue.
ZenRows also provides JavaScript instructions for extracting content that loads dynamically with JavaScript. Armed with these functionalities, you can effectively replace Playwright with ZenRows for all tasks.
Conclusion
You've just learned how to take a screenshot while scraping with Playwright in Python for three use cases:
- Getting a screenshot of the top part of the screen.
- Screenshotting an entire web page, including the parts which require scrolling.
- Capturing the screenshot of a specific element on a web page.
Keep in mind that using Playwright still exposes you to anti-bot systems that can block your screenshotting attempts. By integrating a web scraping API such as ZenRows, you'll bypass all detection mechanisms and scrape any website without getting blocked. Try ZenRows for free!