Using Apify a-sync
Hi there,
I'm using Apify for quiet a while now - great product!
Yet, to get the dataset items of a scraped shop we have only 2 options-"Run actor synchronously and get dataset items" or "Get last run dataset items".
It becomes problematic as long-synchronous-requests could get interrupted, and it looks as if it's only possible to retrieve just the last run results - which means that we can work just on 1 website at a time (no parallelism)
Any work around?
A good option would be to send a request to scrape website X, get the run id Y and then just "get run id (Y) dataset items".
I also checked the "Integrations" - "HTTP Webhook" option but from the available variables it doesn't give access to the dataset items themselves. (<- This would be the best)
Any ideas?
Thanks, Arseni
I'm using Apify for quiet a while now - great product!
Yet, to get the dataset items of a scraped shop we have only 2 options-"Run actor synchronously and get dataset items" or "Get last run dataset items".
It becomes problematic as long-synchronous-requests could get interrupted, and it looks as if it's only possible to retrieve just the last run results - which means that we can work just on 1 website at a time (no parallelism)
Any work around?
A good option would be to send a request to scrape website X, get the run id Y and then just "get run id (Y) dataset items".
I also checked the "Integrations" - "HTTP Webhook" option but from the available variables it doesn't give access to the dataset items themselves. (<- This would be the best)
Any ideas?
Thanks, Arseni
