Apify

Apify Discord Mirror

Updated 5 months ago

Consultant services - Cloudflare Blocked

At a glance

The community member is asking if Apify provides development consultant services. They have a custom Indeed scraper that has been working well, but now they are facing issues with advanced Cloudflare protection. The community member's dev team has rewritten the Actor to use the latest packages and best practices, but the scraper still cannot bypass the protection when deployed to the platform. They have also tried using Apify's recommended hosted Indeed scraper, but are still being blocked.

In the comments, another community member offers to help, and a third community member suggests trying Playwright + Firefox, which they believe should help bypass the protection. They also recommend checking the Apify Academy's anti-scraping documentation for inspiration.

Useful resources
ddanimalweb
·
Does anyone know if Apify provide development consultant services?

We have a custom Indeed scraper, it's been running great for years, but now they have some advanced cloudflare protection in place. Our dev team have rewritten the Actor to use all the latest packages with best practices and recommendations as per the docs.

It runs successfully locally (using proxies) but when deployed to the platform we cannot bypass the protection.

We have also tried to use their recommended hosted indeed scraper and are also blocked.

An example URL would be: https://www.indeed.com/jobs?q=%22advanced+manufacturing%22+or+engineering+or+assembly+or+mechanical+or+automotive&l=New+Orleans%2C+LA&radius=50&jt=fulltime&sort=date
a
d
O
3 comments
aanon_
I could help you out with it
ddanimalweb
How could you help?
OOleg V.

Did you try to use Playwright + firefox?
It should help to bypass protection.

Also, You may find some inspiration here:
https://docs.apify.com/academy/anti-scraping
Add a reply
Sign up and join the conversation on Discord