27 posts tagged with "playwright"

In this article, we will share several ideas on how to download files with Playwright. Automating file downloads can sometimes be confusing. You need to handle a download location, download multiple files simultaneously, support streaming, and even more. Unfortunately, not all the cases are well documented. Let's go through several examples and take a deep dive into Playwright's APIs used for file download.

This guide is a part of the series on web scraping and file downloading with different web drivers and programming languages. Check out the other articles in the series:

• How to download a file with Selenium in Python?
• How to download a file with Puppeteer?
• How to download a file with Playwright?

You open a page in the browser and the data is there. You fetch the same URL with requests, parse it with BeautifulSoup, and the field is empty or shows placeholder text. That is the symptom of a dynamic website: the HTML the server sends is not the HTML you see, because JavaScript changes the page after it loads. This tutorial shows how to confirm that diagnosis on a small fixture and how to get the rendered content with Selenium, Playwright, or the ScrapingAnt API, including the case where the content shows up only after a delay.

Javascript (JS) becomes more popular as a programming language for web scraping. The whole domain becomes more demanded, and more technical specialists try to start data mining with a handy scripting language. Let's check out the main concepts of web scraping with Javascript and review the most popular libraries to improve data extraction flow.

In this article, we will learn how to create a simple e-commerce search API with multiple platform support: eBay and Amazon. AutoScraper and FastAPi provide the ability to create a powerful JSON API for the date. With Playwright's help, we'll extend our scraper and avoid blocking by using ScrapingAnt's web scraping API.

Playwright is a high-level API to control and automate headless Chrome (Chromium), Firefox and Webkit. It can be considered as an extended Puppeteer, as it allows using more browser types to automate modern web apps testing and scraping. Playwright API can be used in JavaScript & TypeScript, Python, C# and, Java. In this article, we are going to show how to set up a proxy in Playwright for all the supported browsers.

Web scraping a website with the actually supported or other browsers has a real benefit in ensuring that the scraper will not be banned by the fingerprint or the behavioral pattern. Playwright already provides full support for Chromium, Firefox, and WebKit out of the box without installing the browsers manually, but since most of the users out there use Google Chrome or Microsoft Edge instead of the open-source Chromium variant, in some scenarios, it's safer to use them to emulate a more realistic browser environment.

In this article, we'd like to share the current state of Playwright integration with Python and several helpful code snippets for understanding the code techniques.