PlaywrightCrawler not competing the jobs I added in the queue using addRequests()
🎭PlaywrightCrawler👨💻Web-Scraping🚀RequestQueue
PlaywrightCrawler not competing the jobs I added in the queue using addRequests().
I am self hosting crawlee and running playwrightCrawler and I add 100-150 requests in queue and and maxComcurrency is 12 and the crawlee don’t finish all the requests that I added. It is never started the crawler for 3 url. I try different urls different websites but it is skipping or whatever but it is no started or no pick 2-4 request from 100-150
I am self hosting crawlee and running playwrightCrawler and I add 100-150 requests in queue and and maxComcurrency is 12 and the crawlee don’t finish all the requests that I added. It is never started the crawler for 3 url. I try different urls different websites but it is skipping or whatever but it is no started or no pick 2-4 request from 100-150
Solution
GitHub
Which package is this bug report for? If unsure which one to select, leave blank @crawlee/playwright (PlaywrightCrawler) Issue description I am experiencing a race condition where the RequestQueue ...
