Set 'ignoreHTTPSErrors' on a PlaywrightCrawler

Hi everyone,

I need to set the

ignoreHTTPSErrors

ignoreHTTPSErrors

flag for a Playwright crawler.

const crawler = new PlaywrightCrawler({
    launchContext: {
        launchOptions: {
            ignoreHTTPSErrors: true,
        }
    },
    requestHandler: router
});

const crawler = new PlaywrightCrawler({
    launchContext: {
        launchOptions: {
            ignoreHTTPSErrors: true,
        }
    },
    requestHandler: router
});

seems to work, but errors on

Type '{ ignoreHTTPSErrors: true; }' is not assignable to type 'LaunchOptions'.\n Object literal may only specify known properties, and 'ignoreHTTPSErrors' does not exist in type 'LaunchOptions'.",

What's the standard way of setting this?
Thanks in advance

MEE6•3/21/23, 1:01 PM

@LeRiton just advanced to level 1! Thanks for your contributions!

Lukas Krivka•3/21/23, 8:29 PM

The type is correct, this option is not there https://playwright.dev/docs/api/class-browsertype#browser-type-launch

BrowserType | Playwright

BrowserType provides methods to launch a specific browser instance or connect to an existing one. The following is a typical example of using Playwright to drive automation:

Lukas Krivka•3/21/23, 8:34 PM

Can toy send me the site, I will test it

HonzaS•3/22/23, 8:57 AM

it is here when you create new context https://playwright.dev/docs/api/class-browser#browser-new-context

standard-azureOP•3/22/23, 9:22 AM

Hi Lukas, Honza, thanks for your answers!
I'm not sure I understand, do you tell me that I need to create a new context to set this flag?
I need this to be set at crawler level, can I define this flag for current browser context and those who will eventually be created?

standard-azureOP•3/30/23, 7:29 PM

@HonzaS @Lukas Krivka can you help me with setting this flag at a crawler level? What am I doing wrong?

HonzaS•3/31/23, 10:34 AM

I would try to change it in the browser context in preNavigationHooks but I am not sure it will work. If you share what site you want to scrape maybe more people can help you.

standard-azureOP•4/4/23, 11:46 AM

I can't share any specific site because we wan't to provide a list of URL for this crawler, some of them in 'http'. This is why we wan't to set this flag at the crawler level.

rubber-blue•4/4/23, 12:30 PM

Could you please try adding

to crawler options? The thing is - has browser launch options, while

ignoreHTTPSErrors

ignoreHTTPSErrors

is new context option, therefore you have to use the preLaunchHooks. I am not sure however whether it's launchContext.ignoreHTTPSErrors or launchContext.launchOptions.ignoreHTTPSErrors

standard-azureOP•4/7/23, 7:53 AM

It works, thanks for this Andrey!

rubber-blue•4/7/23, 5:32 PM

Perfect, glad that it worked out in the end