Suggestion: Adding Batch Execution to the Run API
Currenly the Apify API doesn't support Array as input.
When we request run API with ARRAY input eg:
we get error something like :
Instead error, the API should accept the input as Batch Execution request just like Apollo GraphQL can do: https://www.apollographql.com/docs/router/executing-operations/query-batching/
Advantage:
The Actor only load Docker file once (save load time), then execute Scraper Code multiple time with multiple INPUTS, creating multiple Datasets for each INPUTS.
The run output should also return ARRAY of run results.
When we request run API with ARRAY input eg:
requests.post('https://api.apify.com/v2/acts/actorId/run-sync?token=********', json=[{'location': 'indonesia'}])we get error something like :
{'error': {'type': 'invalid-input', 'message': 'The input JSON must be object, got "array" instead.'}}Instead error, the API should accept the input as Batch Execution request just like Apollo GraphQL can do: https://www.apollographql.com/docs/router/executing-operations/query-batching/
Advantage:
The Actor only load Docker file once (save load time), then execute Scraper Code multiple time with multiple INPUTS, creating multiple Datasets for each INPUTS.
The run output should also return ARRAY of run results.
This is the official developer community of Apify and Crawlee.
14,091Members
Resources
Recent Announcements
Similar Threads
Was this page helpful?
