Can't mark request as failed

Hello,
I am trying to mark a request as failed using crawlee + playwright.

I tried multiple things, from throwing exceptions to use

request.pushErrorMessage()

request.pushErrorMessage()

request.pushErrorMessage()

request.pushErrorMessage() method. The

session.retire()

session.retire()

session.retire()

session.retire() is working for marking the session as bad, but not for mark the request as failed:

await session.retire();
await request.pushErrorMessage(new Error("Captcha detected"));
return;

await session.retire();
await request.pushErrorMessage(new Error("Captcha detected"));
return;

await session.retire();
await request.pushErrorMessage(new Error("Captcha detected"));
return;

await session.retire();
await request.pushErrorMessage(new Error("Captcha detected"));
return;

With the previous code, I get

Terminal status message: Finished! Total 1 requests: 1 succeeded, 0 failed.

Terminal status message: Finished! Total 1 requests: 1 succeeded, 0 failed.

Terminal status message: Finished! Total 1 requests: 1 succeeded, 0 failed.

Terminal status message: Finished! Total 1 requests: 1 succeeded, 0 failed.
When I expect it to fail and retry

Throwing directly an exception also does not work

new Error("Captcha detected")

new Error("Captcha detected")

new Error("Captcha detected")

new Error("Captcha detected")

There is a similar issue solved using Apify actor, but in this case I do not have access to it as I am using vanila Crawlee
https://discord.com/channels/801163717915574323/1105228219013140480/1115869864268152852

rubber-blue•6/7/23, 1:51 PM

You could do something like request.noRetry = true;request.noRetry = true; and then throw after it. If you just throw - request will be retried (3 times by default). This will show request as failed one in the status message

skinny-azureOP•6/7/23, 3:31 PM

Even throwing an exception I still see

Terminal status message: Finished! Total 1 requests: 1 succeeded, 0 failed.

Terminal status message: Finished! Total 1 requests: 1 succeeded, 0 failed.

The only time where It is retrying and I see failed requests is when I have a timeout or the website returns and 403 forbidden. But in my case it is returning an 200 OK but with a captcha and I want to mark this request as failed and retry it.

MEE6•6/7/23, 3:31 PM

@HelThunk just advanced to level 1! Thanks for your contributions!

rubber-blue•6/7/23, 3:33 PM

just throwing an error inside requestHandler will lead to further retry. Retrying 3 times will mark requests as failed and it will be reflected in the status message. I just tried it to double-checked