I want the first urls that I enqueue to run before the next forefront:true.
I want the first urls that I enqueue to run before the next forefront:true.
At a glance
The community member has two sets of URLs that they want to prioritize in their web crawler: the very first URLs that start the crawl, and a set of URLs that should go to the front of the queue on each page. They are having trouble accomplishing this and were considering using two request queues, but encountered issues with that approach.
In the comments, another community member suggests using two request queues and two crawler instances, running the priority queue first and then the other queue. They also mention referring to a comment on the community member's other post.
I have two places that do forefront: true, the very first URLs that I start the crawler with.
Then each page will also have a set of urls that go on the front of the queue, but, I want the very first urls that start the crawl to have priority, how would I accomplish this?
I was thinking of two request queues but had a lot of problems with that as well.
You could use both 2 request queues and 2 crawler instances (each for one of those queues), running first the priority one and then the other one. See also my comment on your other post.
