Apify Discord Mirror

Updated 2 years ago

Resume after crash

At a glance

The community member had a Cheerio crawler running for a few hours, but it crashed. They are wondering if it's possible to renew the crawl from the place it stopped at, as they can see some files left in the key_value_stores directory.

Another community member responded that yes, it is possible to renew the crawl without purging the local stores. They suggested setting the environment variable CRAWLEE_PURGE_ON_START to 0.

The original community member thanked the other for the suggestion.

Additionally, another community member mentioned that they just advanced to level 1 and thanked the original community member for their contributions.

Hey,

I've had a Cheerio crawler running for couple of hours, but it crashed. I'm wondering if it's possible to renew the crawl from the place it stopped at. I can see there are some files left in the
Plain Text
key_value_stores
dir:
Attachment
image.png
L
T
A
3 comments
Hello, yes it is, you just need to start the run without purge local stores. You can set env var CRAWLEE_PURGE_ON_START to 0.
Gotcha, thanks! πŸ™‚
just advanced to level 1! Thanks for your contributions! πŸŽ‰
Add a reply
Sign up and join the conversation on Discord