Concurrent requests and login
Hello,
I’m scraping a website where I have to choose the location (it’s an e commerce store and I have to choose the shop) .
The issue is , I’ve created a post navigation hook that is checking for the location and in case it’s not correct it’s starting the process of selecting the location again, saving the cookies and retry the request with the new cookies.
With this workflow I can only achieve concurrency 1.
Otherwise I’ll have multiple selections of the location at the same time and 99% of time it fails.
How could I somehow pause the entire process, do the selection, and retry all the unhandled requests in the queue .
Thanks !
I’m scraping a website where I have to choose the location (it’s an e commerce store and I have to choose the shop) .
The issue is , I’ve created a post navigation hook that is checking for the location and in case it’s not correct it’s starting the process of selecting the location again, saving the cookies and retry the request with the new cookies.
With this workflow I can only achieve concurrency 1.
Otherwise I’ll have multiple selections of the location at the same time and 99% of time it fails.
How could I somehow pause the entire process, do the selection, and retry all the unhandled requests in the queue .
Thanks !
