How to find the end of all request handlers ?
Hello,
I'm using JSDOMCrawler and inside requestHandler I have some logic for scraping the data and pushing that into an array called messages.
I want to push all these messages to AWS SQS for further processing when the crawlee ends, but I found requestHandler is working asynchronously and due to this I'm not able to get all messages at the end.
Any solution for this scenario ?
I'm using JSDOMCrawler and inside requestHandler I have some logic for scraping the data and pushing that into an array called messages.
I want to push all these messages to AWS SQS for further processing when the crawlee ends, but I found requestHandler is working asynchronously and due to this I'm not able to get all messages at the end.
Any solution for this scenario ?
