Request blocked - received 403 status code

Hello, I'm trying to create a new Actor to scrape Capterra reviews.

It works... 10% of the times. The other 90% I get a 403.

I'm using the default proxyConfiguration and I'm under the free plan.

Any idea how can I bypass the 403?

Thank you so much

Lukas Celnar•7/29/24, 10:42 AM

Hi @jacin
You can try few different things

From my experience using RESIDENTIAL proxies solves the blocking most of the times if at least some requests go through.

Have a look at these documentations that explain how to avoid blocking:

https://docs.apify.com/academy/anti-scraping
https://docs.apify.com/sdk/js/docs/2.3/guides/avoid-blocking

Anti-scraping protections | Academy | Apify Documentation

Understand the various anti-scraping measures different sites use to prevent bots from accessing them, and how to appear more human to fix these issues.

Avoid getting blocked | SDK for JavaScript | Apify Documentation

A Scraper might get blocked for numerous reasons. Let's narrow it down to two main reasons. The first one is a bad or blocked IP address. This topic is covered in the proxy management guide. The second reason we will explore more is browser fingerprints or signatures.