protestant-coral
protestant-coral•2y ago

How to inject storage state into crawler's page

How can a storageState be injected into crawler's page before the page is created? Basically, how to reproduce this but using storageState from the session instead: 
        prePageCreateHooks: [
            async (pageId, browserController, pageOptions) => {
                const storageState = await KeyValueStore.getValue('storageState');
                if (pageOptions && storageState)
                    pageOptions.storageState = storageState;
            },
        ],
        prePageCreateHooks: [
            async (pageId, browserController, pageOptions) => {
                const storageState = await KeyValueStore.getValue('storageState');
                if (pageOptions && storageState)
                    pageOptions.storageState = storageState;
            },
        ],
6 Replies
helpful-purple
helpful-purple•2y ago
Hey @Tobey! Long time no see 🙂 Could you elaborate more on what exactly are you trying to achieve? To be frank - I am not really sure what's happening there :/
protestant-coral
protestant-coralOP•2y ago
Hey @Andrey Bykov ! Indeed. I need to restore storageState from a session and don't know how to do that in a crawler... Ideally, I need access to browserPools context before it is created and before it calls newPage but haven't found a way. I want to pass options to the call to newPage which is the only way to inject the storageState in PW.
protestant-coral
protestant-coralOP•2y ago
rare-sapphire
rare-sapphire•2y ago
Another apify alumni, bring beers 😄 I was trying to hijack newPage too recently, and found this old discussion mentioning that modifying finalPageOptions works only with useIncognitoPages: true And indeed, it works! (didn't have time to investigate why it is, but that's the world we live in 😄 ) https://discord.com/channels/801163717915574323/1018881795791532093/1019020911468359731
helpful-purple
helpful-purple•2y ago
@strajk and the thread you pointed brings another apify alumni 😄
protestant-coral
protestant-coralOP•2y ago
Cheers! PW ignores the options passed from browserPool unless in incognito mode. Not sure why, looks like a bug to me... the code there is quite messy. I think there should be a way to intercept the browserPool calls to newContext() and inject the options before it's created but didn't find any. Do you know of any way?

Did you find this page helpful?