fascinating-indigo•2y ago
I want the first urls that I enqueue to run before the next forefront:true.
I have two places that do forefront: true, the very first URLs that I start the crawler with.
Then each page will also have a set of urls that go on the front of the queue, but, I want the very first urls that start the crawl to have priority, how would I accomplish this?
I was thinking of two request queues but had a lot of problems with that as well.
1 Reply
adverse-sapphire•2y ago
You could use both 2 request queues and 2 crawler instances (each for one of those queues), running first the priority one and then the other one. See also my comment on your other post.