xenial-black•2y ago

Efficiency / Cost management of multiple smaller actors vs. one large actor

I'm using the clockworks tiktok scraper and had dilemna on whether to use 8 actors with 4gb RAM each, spreading out pages to scrape evenly accross the actors vs. 1 large 32gb actor for all the pages. It seems like from trying out the scrape its far cheaper (not much faster though), to spread out the jobs accross 8 actors (we're looking at $10-12 vs. $3-4). I was adviced from the team to try one large actor since it should scale okay, but seems like my findings are different? Am I doing something wrong here, or is it just that this scrape is better with me batching the jobs.