conscious-sapphire•3y ago

How to solve example.com needs to review the security of your connection before proceeding?

I am trying to scrape a site that has a page that check if the connection is secure and I get this error: WARN PlaywrightCrawler: Reclaiming failed request back to the list or queue. Request blocked - received 403 status code.

5 Replies

conscious-sapphireOP•3y ago

I think it needs to wait longer, instead of to shut down after 1 second and to try again, because it always get the error, how can this be done?

Lukas Krivka•3y ago

This is a Cloudflare check for a real browser usually. Try to wait longer, use headless: false and Firefox

conscious-sapphireOP•3y ago

Is there a guide on how to use Firefox?

adverse-sapphire•3y ago

@pelsec check out this example: https://crawlee.dev/docs/examples/playwright-crawler-firefox

conscious-sapphireOP•3y ago

Thank you

How to solve example.com needs to review the security of your connection before proceeding?

Did you find this page helpful?