Is the default request queue the same for different crawler instances?
Hello everyone,
I would like to know if the default request queue (if not specified in the Crawler options) is the same for all instances?
I tried to run an HttpCrawler next to a PlaywrightCrawler and for some unknown reason the HttpCrawler picked a request which was for the PlaywrightCrawler
Thanks
I would like to know if the default request queue (if not specified in the Crawler options) is the same for all instances?
I tried to run an HttpCrawler next to a PlaywrightCrawler and for some unknown reason the HttpCrawler picked a request which was for the PlaywrightCrawler
Thanks
Solution
Yes, I believe. If you don't specify a queue's name, they both would use the same default queue. Solution is to use named queues that you'd drop at the end of the actor run
See: https://github.com/apify/crawlee/discussions/2026?utm_source=chatgpt.com#discussioncomment-6656135
See: https://github.com/apify/crawlee/discussions/2026?utm_source=chatgpt.com#discussioncomment-6656135
