How to set ram used by the crawler
Ive scoured the docs and used chatgpt/perplexity. I for the life of me cannot work out how to set the ram available to the crawler. I want to give it 20gb i have a 32gb system
HSomeone will reply to you shortly. In the meantime, this might help:
ELearn the Apify platform's resource capability and limitations such as max memory, disk size and number of Actors and tasks per user.
WIm not using Apify, is this not place for crawlee python questions?
Eare you going to set the RAM in your local python code?
EIn most case, we discuss Apify actor crawlee here
MCrawlee helps you build and maintain your Python crawlers. It's open source and modern, with type hints for Python to help you catch bugs early.
WHi so I’ve found that but clearly I am not a good developer because I haven’t figured out what I actually need to write in the code to use it.
Doesn’t work for me.
Doesn’t work for me.
MIs it not working when used in this way?
WThats the correction I was probably looking for. Ive just tried it on my macbook and it starts crawling w/o errors. When I get home Ill try it on my actual computer and let you know how it goes
M@Grespino just advanced to level 1! Thanks for your contributions! 
Wok so it works in the script but im not quite sure if its working properly. It doesnt seem to going faster althought that could be because I am only trying to do one domain atm rather than many in parallel?
here s the autoscaling stuff from console:
[crawlee._autoscaling.autoscaled_pool] INFO current_concurrency = 25; desired_concurrency = 200; cpu = 0.0; mem = 0.0; event_loop = 0.295; client_info = 0.0
here s the autoscaling stuff from console:
[crawlee._autoscaling.autoscaled_pool] INFO current_concurrency = 25; desired_concurrency = 200; cpu = 0.0; mem = 0.0; event_loop = 0.295; client_info = 0.0
MMergers expect PR related to parallelism - https://github.com/apify/crawlee-python/pull/780
This may be related to the problem it solves.
This may be related to the problem it solves.