Skip to main content

How to use a proxy in Playwright

Oleg Kulyk

Oleg Kulyk

Co-Founder @ ScrapingAnt

How to use a proxy in Playwright?

Playwright is a high-level API to control and automate headless Chrome (Chromium), Firefox and Webkit. It can be considered as an extended Puppeteer, as it allows using more browser types to automate modern web apps testing and scraping. Playwright API can be used in JavaScript & TypeScript, Python, C# and, Java. In this article, we are going to show how to set up a proxy in Playwright for all the supported browsers.

How to use rotating proxies with Puppeteer

Oleg Kulyk

Oleg Kulyk

Co-Founder @ ScrapingAnt

How to use rotating proxies with Puppeteer?

Puppeteer is a high-level API to control headless Chrome. Most things that you can do manually in the browser can be done using Puppeteer, so it quickly became one of the most popular web scraping tool in Node.js and Python. Many developers use it for a single page applications (SPA) data extraction as it allows executing client-side Javascript. In this article, we are going to show how to set up a proxy in Puppeteer and how to spin up your own rotating proxy server.

How to use Microsoft Edge with Playwright

Oleg Kulyk

Oleg Kulyk

Co-Founder @ ScrapingAnt

How to use Microsoft Edge with Playwright

Web scraping a website with the actually supported or other browsers has a real benefit in ensuring that the scraper will not be banned by the fingerprint or the behavioral pattern. Playwright already provides full support for Chromium, Firefox, and WebKit out of the box without installing the browsers manually, but since most of the users out there use Google Chrome or Microsoft Edge instead of the open-source Chromium variant, in some scenarios, it's safer to use them to emulate a more realistic browser environment.

HTML Parsing Libraries - C#

Oleg Kulyk

Oleg Kulyk

Co-Founder @ ScrapingAnt

HTML Parsing Libraries - C#

Web sites are written using HTML, which means that each web page is a structured document. Sometimes the goal is to obtain some data from them and preserve the structure while we’re at it. Websites don’t always provide their data in comfortable formats such as CSV or JSON, so only the way to deal with it is to parse the HTML page.

HTML Parsing Libraries - JavaScript

Oleg Kulyk

Oleg Kulyk

Co-Founder @ ScrapingAnt

HTML Parsing Libraries - JavaScript

HTML is a simple structured markup language and everyone who is going to write the web scraper should deal with HTML parsing. The goal of this article is to help you to find the right tool for HTML processing. We are not going to present libraries for more specific tasks, such as article extractors, product extractors, or web scrapers.