Vverbal-lime
Abort Crawler on Exception
Hello everyone! I'm trying to implement a scraper based on puppeteer. My logic looks like as simple as below:
What I'm trying to achieve is: when there's any exception thrown for
However, it looks like when there's an exception for
Is there any configuration that I can do to achieve this?
What I'm trying to achieve is: when there's any exception thrown for
url_1url_1 (e.g. http error code or any other exceptions inside the request handler), an exception will be also thrown after the first line, so that url_2url_2 won't be scraped.However, it looks like when there's an exception for
url_1url_1, Crawlee handles it gracefully, and continues to execute the second line there. I searched around the doc, github issues, and this channel for a while but didn't find any luck.Is there any configuration that I can do to achieve this?
