sunny-green•2y ago
Is there a way to use both PlaywrightCrawler and CheerioCrawler?
I am getting PDF links using PlaywrightCrawler. I would like to download those PDF links, but when I use
sendRequest with it, I got the error waiting until "load". I suspect this is because of PlaywrightCrawler, it seems that I need to use CheerioCrawler? Is this correct? If so, how do I use both?3 Replies
sunny-greenOP•2y ago
I found out the error. It was because Playwright was waiting for navigation. I should use the
skipNavigation instead
but I am curious, if there is actually away to use different kinds of crawlers at one runUsually you want to use one crawler after the other. If you use them in parallel, you need to assign them unique queues and storage so their state doesn't clash.
sunny-greenOP•2y ago
Oh okay, interesting. Thanks Lukas