The community member is experiencing an issue where a website is blocked when accessed through the crawlee tool and a proxy, but can be accessed without any issues on the same system, other browsers, and in incognito mode. The comments suggest that the website may be checking the hardware or detecting an automated browser, and provide some potential solutions such as trying Playwright with different browsers, running a headful browser, and using different proxies like residential proxies.
I have a weird situation. whenever I try to access a website via crawlee and with proxy the request is blocked but with the same proxy I can access the website without any problem on my system and with many other browsers and also in incognito mode. Its really puzzling me. Any help would be highly appreciated. Thankyou.
If by on my system you mean regular browser then there is nothing weird about that. The site just detects that you are using automated browser and blocks the request.
