rising-crimson
rising-crimson•2y ago

how to see if enqueueLinks are found or not

await enqueueLinks({
        strategy: EnqueueStrategy.SameDomain,
        globs: globs,
        userData: request.userData,

    });
await enqueueLinks({
        strategy: EnqueueStrategy.SameDomain,
        globs: globs,
        userData: request.userData,

    });
there are many urls given to the crawler. I want to initiate another requestHandler when it's done crawling all the relatives urls of one domain. How can I check if the crawler has stopped crawling one domain?
1 Reply
Lukas Krivka
Lukas Krivka•2y ago
You would need to track it yourself in the state object. You can add all requests that were enqueued (using the transformRequestFunction) and then update the object when a request is processed

Did you find this page helpful?