passive-yellow•3y ago
Request queue with id Error
Hello, how to manage multiple instances of puppeteer running at the same time. If i launch one browser - everything works fine, but while crawling the first link, if i try to launch one more instance - i receive
Request queue with id: 63925d40-c5fa-4b8d-a1eb-a4f7b93a5253 does not exist.
The same problem occurs on localhost and ubuntu prod server
2 Replies
passive-yellowOP•3y ago
More code from my project
Hello,
Generally running multiple paralel crawlers inside one Actor is not a good idea, since they share might share the same resource, which I believe is also your case. You might need to assign a different RequestQueue to each of these crawler instances, but I thing this way you will run into another problems soon.
What is the motivation for runing multiple crawlers in parallel?