ratty-blush
ratty-blush•3y ago

Add label to pages via `crawler.addRequests()`?

I am adding a page as the initial crawl target, but would like to add a label to ensure it routes to the correct processor. Is there a way to do this? 
await crawler.addRequests([  "https://www.foo.bar/page",
])
await crawler.addRequests([  "https://www.foo.bar/page",
])
It seems I can only add RequestQueueOperationOptions with addRequests, not the same EnqueueLinksOptions that I can use with enqueueLinks().
2 Replies
NeoNomade
NeoNomade•3y ago
Create a Request object and put it into your array . In the request object you can use label
correct-apricot
correct-apricot•3y ago
await crawler.addRequests([{
    url: 'https://www.foo.bar/page,
    label: 'start-url',
}]);
await crawler.addRequests([{
    url: 'https://www.foo.bar/page,
    label: 'start-url',
}]);

Did you find this page helpful?