addRequestsBatched isn’t working for me on Apify, even though it works perfectly fine locally
Hi, I’m having trouble with
addRequestsBatched (& also addRequests) It works perfectly fine when I run it locally, but it on Apify it does not seem to do anything. Has anyone faced this issue or knows what might be causing it?
I'm using the exact same versions of node (22.19.0) , crawlee (3.15.1) & apify (3.4.5)4 Replies
I had the same problem when using a small amount of memory. Try increasing the amount of memory.
Or it could be an IP blocking issue.
When it came to IP blocking, I solved the problem by using Apify Datacenter Proxy.
Thanks for your suggestions, but it's definitely not an IP blocking thing, nor a memory issue (I'm on 1GB of RAM & my actor only does HTTP requests, highest usage was 99MB)
Anyway, I refactored the actor to not use crawlee and it's working for me but no clue why adding to the queue didn't.
Got it, thanks for clarifying. Interesting that removing Crawlee fixed it — that suggests the issue may have been related to how the request queue was being managed internally (e.g. concurrency, auto-retries, or how requestQueue.addRequests() was being awaited). Sometimes if requests are added without proper await, or if the queue is closed/finished too early, they never actually get processed.
Thanks Exp. I'm awaiting it & it's literally a single request that I'm enqueing.. anyway..