eastern-cyan•17mo ago
web Crawler got blocked in selenium
message like Verifying you are human. This may take a few seconds.
In robots.txt
User-agent: *
Disallow: /
please share your suggestion on above problem
2 Replies
@Umang dm me
hi, try to check out this article for more info https://docs.apify.com/academy/anti-scraping
Anti-scraping protections | Academy | Apify Documentation
Understand the various anti-scraping measures different sites use to prevent bots from accessing them, and how to appear more human to fix these issues.