Cloudflare bypass fingerprints

Dear all, I am currently experimenting to bypass cloudflare Security. I am using playwright but getting dectected. My ip is whitelisted. Can any one of you please guide me how can I create a fingerprint which let me bypass cloudflare ?

Thankyou in advance.

ordinary-sapphire•3/5/23, 10:39 AM

use proxies?

military-pinkOP•3/5/23, 10:41 AM

But if my ip is already whitelisted then what difference a proxy could make ?
In prod I will for sure use proxies but for testing am using my residential internet ip.

rubber-blue•3/5/23, 11:58 AM

Are you using just playwright or crawlee playwright crawler? Crawlers already have fingerprints enabled by default. But may also want to try full (non-headless mode) or playwright + firefox combination

military-pinkOP•3/5/23, 12:31 PM

Yes am using crawlee playwright. What configuration I need to change to run in full mode and with firfox ? Any link to docs ?

rubber-blue•3/5/23, 12:33 PM

firefox: https://crawlee.dev/docs/examples/playwright-crawler-firefox
headless (should be set to false): https://crawlee.dev/api/playwright-crawler/interface/PlaywrightCrawlerOptions#headless

military-pinkOP•3/5/23, 2:59 PM

Thanks for the links. I think know the problem. Crawlee doesnt let the cloudflare complete the bot detection. It launches the browser, and before even cloudflare could check it, crawlee just close the browser.

Anyway to avoid this behaviour ?

MEE6•3/5/23, 2:59 PM

@curioussoul just advanced to level 1! Thanks for your contributions!

rubber-blue•3/5/23, 3:47 PM

oh yesss, I completely forgot about it. it should be addressed in one of the next releases - there apparently was some change in how cloudflare works. For now the fastest way to solve it would be to add to your crawler options

military-pinkOP•3/5/23, 7:48 PM

Thanks for the help. It worked now. Its bypassing CF secuirty in headless mode as well now.

Great work guys. I get to know about crawlee few days ago and today I tried it and its amazing. Lots of respect for you guys.

rubber-blue•3/5/23, 7:58 PM

Thanks for the feedback

Trying our best