Sspotty-amber
Concept of a 'queue'?
I think i am confused on the request queue.
Here's my scenario: I will be having many requests for the same actor, where I will pass a URL. I don't know the frequency of these requests.. I may have several in a few minutes, or several in an hour.. and they may take 2-10 minutes to run.
Should I be creating a new run each time I want a request? I know I can START a run with many URL's.. But I may only have one to add at a time.
I'm concerned I will start 10x parallel runs, and crash my memory.
What's the best practice? Can I just keep a single run going, and pass all URL's to this run?
Here's my scenario: I will be having many requests for the same actor, where I will pass a URL. I don't know the frequency of these requests.. I may have several in a few minutes, or several in an hour.. and they may take 2-10 minutes to run.
Should I be creating a new run each time I want a request? I know I can START a run with many URL's.. But I may only have one to add at a time.
I'm concerned I will start 10x parallel runs, and crash my memory.
What's the best practice? Can I just keep a single run going, and pass all URL's to this run?
This is the official developer community of Apify and Crawlee.
14,091Members
Resources
Recent Announcements
Similar Threads
Was this page helpful?
