Ssacred-rose
Prevent Clawler from adding failed request to default RequestQueue
Is there a way to prevent the crawler from adding a failed request to the default RequestQueue?
I'm using the default
I'm using the default
RequestQueueRequestQueue to add productUrls, and they're being handled inside the defaultRequestHandlerdefaultRequestHandler, but when some of them fails, I purposely throw an Error, expecting the failed request(which is the initUrlinitUrl) goes back to RequestListRequestList, but it goes to the default RequestQueueRequestQueue too, which is not what I want.
