Skip to main content

How to avoid IP rate limits

Oleg Kulyk

Oleg Kulyk

Co-Founder @ ScrapingAnt

How to avoid IP rate limiting?

Web scraping specialists are dealing with using proxy servers to overcome various anti-bot defenses every day. One of those protections is IP rate limiting, a primary anti-scraping mechanism.

Let's learn more about this protection method and the most effective ways of bypassing it.

Best Free Proxy Scraping Tools

Oleg Kulyk

Oleg Kulyk

Co-Founder @ ScrapingAnt

Best open source proxy scrapers

Using a quality proxy server is the key to a successful web scraper. A variety of IPs along with their quality make it possible to collect data from various web sites without worrying about being blocked.

Still, many websites provide free proxy lists, so can the process of getting IP addresses from them be automated? Are free proxies good enough for web scraping? Let's check it out.

How to parse HTML in .NET

Oleg Kulyk

Oleg Kulyk

Co-Founder @ ScrapingAnt

How to parse HTML in .NET

HTML parsing is a vital part of web scraping, as it allows convert web page content to meaningful and structured data. Still, as HTML is a tree-structured format, it requires a proper tool for parsing, as it can't be property traversed using Regex.

This article will reveal the most popular .NET libraries for HTML parsing with their strong and weak parts.

Web Scraping with Java

Oleg Kulyk

Oleg Kulyk

Co-Founder @ ScrapingAnt

Web Scraping with Java

Java is one of the most popular and high demanded programming languages nowadays. It allows creating highly-scalable and reliable services as well as multi-threaded data extraction solutions. Let's check out the main concepts of web scraping with Java and review the most popular libraries to setup your data extraction flow.

How to download a file with Playwright?

Oleg Kulyk

Oleg Kulyk

Co-Founder @ ScrapingAnt

How to download a file with Playwright?

In this article, we will share several ideas on how to download files with Playwright. Automating file downloads can sometimes be confusing. You need to handle a download location, download multiple files simultaneously, support streaming, and even more. Unfortunately, not all the cases are well documented. Let's go through several examples and take a deep dive into Playwright's APIs used for file download.