postNavigationHooks timeout
I'm using camoufox and handleCloudflareChallenge utility in postNavigationHooks and request timeout after 100 seconds. Is it possible to lower the timeout limit from 100 in postNavigationHooks? it seems like it doesn't respect requestHandlerTimeoutSecs or navigationTimeoutSecs
Solution:Jump to solution
RequestHandlerTimeoutSecs is enforced by Apify/Crawlee’s overall request handler, but once inside a postNavigationHook, you're in user-defined logic.
If handleCloudflareChallenge doesn't internally support a timeout (or ignores one), it might block longer than desired.
navigationTimeoutSecs applies to page.goto() and similar calls — not necessarily to post-navigation scripts....
2 Replies
Solution
RequestHandlerTimeoutSecs is enforced by Apify/Crawlee’s overall request handler, but once inside a postNavigationHook, you're in user-defined logic.
If handleCloudflareChallenge doesn't internally support a timeout (or ignores one), it might block longer than desired.
navigationTimeoutSecs applies to page.goto() and similar calls — not necessarily to post-navigation scripts.
yeah, I checked the docs to see if Crawlee has something for this, but I couldn’t find anything. I figured I’d ask here in case I missed it but I guess it’s not there, so I’ll have to write the logic myself