Max requests per second

Hello!

I would like to know, is there, like for the maxRequestsPerMinute / maxTasksPerMinute an option but for second?
If not, what would be the easiest way to implement this? Always waiting 1s in the request handler and relying on maxConcurrency?

Thanks!

MEE6•6/1/25, 1:02 PM

@MrSquaare just advanced to level 1! Thanks for your contributions!

lemurio•6/6/25, 3:05 PM

hey, there is currently no option for second, you could consider setting maxConcurrency to some reasonable number.
this article could help: https://crawlee.dev/js/docs/guides/scaling-crawlers

Scaling our crawlers | Crawlee for JavaScript · Build reliable cra...

Crawlee helps you build and maintain your crawlers. It's open source, but built by developers who scrape millions of pages every day for a living.