Ppopular-magenta
Is it possible to stop the crawler if a condition is met ?
Hi. I'm making a crawler (CheerioCrawler) that scrape a news website. I start the crawler by giving it a list of url with all the pages of the articles list (an array containing site.com/?page=1, site.com/?page=2 etc). For every article list page, i will scrape every article inside of it. I was wondering, if my url site.com/?page=60 (per instance) doesn't have any articles on it, can i stop the execution of the crawler at this time ?
I know how to check if there are any articles on the page, but I can't find how to stop the crawler at a certain point (without completing all the url in the url list).
Thank you very much!
I know how to check if there are any articles on the page, but I can't find how to stop the crawler at a certain point (without completing all the url in the url list).
Thank you very much!
This is the official developer community of Apify and Crawlee.
13,923Members
Resources
Recent Announcements
Similar Threads
Was this page helpful?
