harsh-harlequin•2y ago
CheerioCrawler only scrapes one url
Hi, I'm trying to scrape https://www.noom.com but it crawls only that url but not all urls available that are in this page, I added retryOnBlocked: true and played with all EnqueueStrategy but it still the same, it happens on this website only.
Is there a way to make it work? thank you
4 Replies
You need to save web output https://docs.apify.com/academy/node-js/analyzing-pages-and-fixing-errors#saving-snapshots and check you you actually getting under scraper
harsh-harlequinOP•2y ago
Right I guess the CheerioCrawler doesn’t handle screenshots?
It works fine now that I've replace CheerioCrawler with PlaywrightCrawler
@miky2fois just advanced to level 1! Thanks for your contributions! 🎉
cheerio only processing html response, so there is no "screen" to capture under it, since there is no browser