worthy-azureW
Apify & Crawlee3y ago
1 reply
worthy-azure

Is it preferable to keep a crawler(s) alive or keep spawning and tearing them down?

Hi, I am using puppeteer crawler to crawl my site to take snapshots for a visual regression testing system. There is a CI pipeline that will add a bunch of urls to crawl. Is there any advantage of just keeping a crawler running and waiting for requests to enter the queue?
Was this page helpful?