ordinary-sapphireO
Apify & Crawlee•3y ago•
5 replies
ordinary-sapphire

Avoid sharing same CheerioCrawler instance across multiple calls

Hi, I'm initializing a 
new CheerioCrawler()
 when ever a POST request is received by my server, but even though I make a new instance of CheerioCrawler for each request, they seem to be affected by configs like 
maxRequestsPerCrawl: 20
 from previous requests. I've tried to do 
await crawler.teardown();
 but to no avail. What's even worse is that for two requests that come in at the same time, the 
requestHandler
 is polluted by the separate calls! How do I isolate instances of 
CheerioCrawler
 from each other?
Was this page helpful?