Gracefully closing the crawler with keepalive flag true
I'm using puppeteer crawler with
This runs the crawler infinitely and if I insert new requests to the requests queue, they get processed.
(I'm using non persisted request queue)
What I want is to gracefully close the crawler, as in If I get a signal to close, I want to process all the pending requests in the requests queue first and then kill the crawler.
Right now If I do
keepAlivetruecrawler.run()This runs the crawler infinitely and if I insert new requests to the requests queue, they get processed.
(I'm using non persisted request queue)
What I want is to gracefully close the crawler, as in If I get a signal to close, I want to process all the pending requests in the requests queue first and then kill the crawler.
Right now If I do
crawler.teardown()
