Is there a gentle way to run a crawler for x minutes?
Is there a gentle way to run a crawler for x minutes?
At a glance
The community member is looking for a way to stop crawling after a certain number of requests using the maxRequestsPerCrawl option. Another community member suggests there may be a "gentle" way to run a crawler for a specific duration of time. A third community member provides a link to Apify's documentation, which mentions the ability to set a timeout, but the original poster clarifies they are not using Apify and are instead using crawlee.dev locally. There is no explicitly marked answer in the comments.