Monitoring CPU and memory usage in actors

At a glance

The community member is developing an actor in Rust and trying to access CPU and memory utilisation. They found that Crawlee uses os.cpus() in Node.js, but they are looking for a Rust equivalent. The community member can make it work locally by mounting the Docker socket, but it doesn't work on the Apify platform.

Another community member points out that the Apify documentation has information about system events, including cpuInfo and systemInfo events that provide CPU and memory usage data. However, the documentation is missing some details, such as the total available CPU and memory, which the community member needs to calculate usage percentages.

The community members discuss how to find the total available CPU and memory, with one community member noting that the total memory is available in an environment variable, and the number of CPU cores depends on the memory allocation. The remaining question is the unit of the cpuCurrentUsage value.

There is no explicitly marked answer in the comments, but the community members continue to discuss and share information to help resolve the issue.

Useful resources

JJerome

Hi, I'm develping an actor in rust and I'm trying to access resources utilisation (CPU and memory). I've seen that Crawlee uses os.cpus() from node, but I'm looking for a rust equivalent.
I can make it work locally by mounting the docker socket on the container (docker run -v /var/run/docker.sock:/var/run/docker.sock <image_tag>), but it does not work on the Apify platform.
Are there any resources/pointers I could check on how Apify runs an actor's container? And how I could read these resources utilisation?
Any help would be appreciated.

7 comments

AApifyBot

just advanced to level 1! Thanks for your contributions! 🎉

JJerome

Turns out I had missed this page from the docs: https://docs.apify.com/platform/actors/development/programming-interface/system-events.

The websocket sends 2 events (the second one is not documented though):

Plain Text

{
   "name":"cpuInfo",
   "data":{
      "isCpuOverloaded":false,
      "createdAt":"2024-06-05T09:16:37.916Z"
   }
}

Plain Text

{
    "name": "systemInfo",
    "data": {
        "memAvgBytes": 127389235.71674345,
        "memCurrentBytes": 168525824,
        "memMaxBytes": 257417216,
        "cpuAvgUsage": 1.655466795311761,
        "cpuMaxUsage": 115.72125786163522,
        "cpuCurrentUsage": 7.616518557239937,
        "isCpuOverloaded": false,
        "createdAt": "2024-06-05T09:16:39.092Z"
    }
}

vvojtechmaslan

Hello , thanks for pointing out the missing documentation. I will pass it on the team so we can fix it.

JJerome

Hi , thanks that'd be great.
What seems to be missing from these stats is the total amount of available CPU/memory, is there a way to get that?
The goal would be to compute a usage % to be able to throttle the system.
Also I have no idea what "cpuCurrentUsage": 7.616518557239937 means, what unit is that?
Thanks for your help.

JJerome

Ok total amount of memory is given by env variable ACTOR_MEMORY_MBYTES, and number of CPU cores depends on the memory: "1 GB gives you 0.25 CPU cores".

So the only question remaining is what unit is the cpu usage in?

ccoinmaster47

how do you find total CPU/RAM usuage?

ccoinmaster47

found it: https://api.apify.com/v2/acts?token=xxxx

Add a reply

Apify Discord Mirror

Monitoring CPU and memory usage in actors