fascinating-indigo•2y ago
How to change page options on error?
Hey 👋 I'm wondering how could I open a playwright new page with changed options only for specific request?
Example scenario:
1. request fails so it goes to error handler
2. error handler puts the request on request queue
3. in
prePageCreateHooks if this is the failed request, change for example isMobile option in pageOptions (or any option defined here https://playwright.dev/docs/api/class-browser#browser-new-page)
4. proceed with this page in requestHandler
I've searched in the code if it's possible and it looks like context with a request is available when creating a page: https://github.com/apify/crawlee/blob/193432c13308599b6761ad2655c48e1ccd5f50d1/packages/browser-crawler/src/internals/browser-crawler.ts#L514
But hooks for page creation doesn't have access to this context: https://github.com/apify/crawlee/blob/193432c13308599b6761ad2655c48e1ccd5f50d1/packages/browser-pool/src/browser-pool.ts#L526
Is there any way to handle such scenario? Or maybe should I catch an error and create a new page myself in requestHandler?1 Reply
Suggested approach is to resolve all the above by handler function, i.e. if you not getting data from request1 then add request2 with changes and process it as a new. No good reason to retry request1, since purpose is literally retrying to resolve random blocking.