absent-sapphire•3y ago
Get data old link crawler
When I use PuppeteerCrawler, I run the same URL for crawling, I want to get previous data crawled of this link
1 Reply
stormy-gold•3y ago
Purging of storages is happening automatically when you run your crawler.
You can disable it that via
purgeOnStart Configuration option or by setting CRAWLEE_PURGE_ON_START environment variable to 0 or false.
https://crawlee.dev/docs/guides/configuration#crawlee_purge_on_startConfiguration | Crawlee
Configuring Crawlee parameters