correct-apricot•3y ago
Stoping Crawler when done in scraping
Good day everyone, How can I make the crawler stop? When its done to scraper/request a certain url?
Because I want to setup my crawlee project that continously running when it has no url to request, like waiting to a queue of urls (redis)
2 Replies
pleasant-yellow•3y ago
CriticalError | API | Crawlee
Errors of
CriticalError type will shut down the whole crawler.correct-apricotOP•3y ago
How can I do that? throw an CriticlError even its not encountering error?