MPower Automate known issues?
GeneralHi, just asking around if any of your run your actor requests through Power Automate? If so, do you know of any issues that cut off the responses (the actor I used returned all 3000 posts in the Apify console, but only 29 showed up on Power Automate)?
I've ruled out that adjusting the pagination count should prevent the issue from recurring but I still want to make sure. Thanks all.
I've ruled out that adjusting the pagination count should prevent the issue from recurring but I still want to make sure. Thanks all.
Solution
Yeah, I think this is probably a pagination or response-size issue on the Power Automate side.
For example, if the Actor successfully scraped 3,000 posts and I can see all 3,000 in the Apify dataset, then the Actor itself is probably working correctly.
Instead of asking Power Automate to retrieve all 3,000 posts at once, I would retrieve them in smaller batches. For example, I could get 500 posts first with offset zero, then another 500 with offset 500, then 1,000, 1,500, and so on.
So basically, I would keep increasing the offset until there are no more results.
I would also check the Power Automate run history. For example, if the Apify API response contains 500 posts but only 29 reach the next Power Automate step, then I know the problem is happening inside the Power Automate flow rather than Apify.
So yes, I think adjusting the pagination is the right approach, but I’d also check the raw response and the Apply to each step to find exactly where those records are getting cut off.
For example, if the Actor successfully scraped 3,000 posts and I can see all 3,000 in the Apify dataset, then the Actor itself is probably working correctly.
Instead of asking Power Automate to retrieve all 3,000 posts at once, I would retrieve them in smaller batches. For example, I could get 500 posts first with offset zero, then another 500 with offset 500, then 1,000, 1,500, and so on.
So basically, I would keep increasing the offset until there are no more results.
I would also check the Power Automate run history. For example, if the Apify API response contains 500 posts but only 29 reach the next Power Automate step, then I know the problem is happening inside the Power Automate flow rather than Apify.
So yes, I think adjusting the pagination is the right approach, but I’d also check the raw response and the Apply to each step to find exactly where those records are getting cut off.
