Apify Discord Mirror

Updated 5 months ago

Cheerio memory error

At a glance

The community member has deployed a CheerioCrawler on AWS with 2vCPU and 4GB of RAM, but is experiencing a memory overload issue. The error message indicates that the crawler is using 1174 MB of 750 MB (157%) of available memory. Another community member suggests setting the APIFY_MEMORY_MBYTES environment variable to address the issue. Additionally, a solution is provided by setting the availableMemoryRatio configuration to 0.95, which appears to have fixed the problem.

Useful resources

NNeoNomade

Hello,
I have deployed a CheerioCrawler on AWS, the machine has 2vCPU and 4gb of ram, but I get the following error:

Plain Text

WARN CheerioCrawler:AutoscaledPool:Snapshotter: Memory is critically overloaded. Using 1174 MB of 750 MB (157%). Consider increasing available memory.

What can it be ?

2 comments

PPepa J

Hello have you tried setting up and enviromental variable APIFY_MEMORY_MBYTES ? https://docs.apify.com/sdk/js/docs/guides/environment-variables#apify_memory_mbytes

NNeoNomade

fixed it like this:

Plain Text

const config = Configuration.getGlobalConfig();
config.set('availableMemoryRatio', 0.95)

Add a reply