@Lukas Krivka hello, could u help answer

@Lukas Krivka hello, could u help answer this? i tried searching documentation on OOM but they arent clear about the conditions which trigger OOM when im running playwright via nodejs so i can preempt oom error. i have minimized memory leaks on nodejs side as much as i can by garbage collection

Lukas Krivka•9/26/25, 9:36 AM

Sadly, there isn't any bullet proof solution. Like in every server you run, you can run out of memory.

What happens is that Crawlee will not scale up if it is getting close to max memory but if there is some huge random spike (like you parsed some huge HTML page), it can still happen. You can do 2 things:

Check that you are not storing any big data into your memory objects. Ideally the handlers just push data and you don't keep anything in memory long term.
Slow down scaling, look at the autoscaledPoolOptions, you can make it slow down going up and be more worried about memory rate.