RóiR
Apify & Crawlee7mo ago
3 replies
Rói

Configuring playwright + crawlee js to bypass certain sites

I have noticed some pages that appear completely normal are sometimes hard to fetch content from. After some investigation, it might have something to do with the site being behind cloudflare. Do you have any suggestions on how to get past this? I believe in certain cases, it's simply a matter of popups and accepting some cookies. I do have stealth plugin added, but it still does not pierce through.
Was this page helpful?