BrowserPoolOptions
Hello i have browserPoolOptions in PlaywrightCrawler like this:
browserPoolOptions: {
maxOpenPagesPerBrowser: 0,
useFingerprints: true,
preLaunchHooks: [async (pageId, launchContext) => {
launchContext.launchOptions = {
...launchContext.launchOptions,
// eslint-disable-next-line max-len
userAgent: 'Mozilla/5.0 (Linux; U; Android 3.2; nl-nl; GT-P6800 Build/HTJ85B) AppleWebKit/534.13 (KHTML, like Gecko) Version/4.0 Safari/534.13',
bypassCSP: true,
ignoreHTTPSErrors: true,
hasTouch: true,
isMobile: true,
deviceScaleFactor: 1,
};
}],
},
and when I run this scraper, it will start with scraping page by page but the problem is that the scraper doesn't close craped page so every window is still open so after while actor reach crytical level of memory consuption and fall down.
Do you have any ideas how to fix it?
browserPoolOptions: {
maxOpenPagesPerBrowser: 0,
useFingerprints: true,
preLaunchHooks: [async (pageId, launchContext) => {
launchContext.launchOptions = {
...launchContext.launchOptions,
// eslint-disable-next-line max-len
userAgent: 'Mozilla/5.0 (Linux; U; Android 3.2; nl-nl; GT-P6800 Build/HTJ85B) AppleWebKit/534.13 (KHTML, like Gecko) Version/4.0 Safari/534.13',
bypassCSP: true,
ignoreHTTPSErrors: true,
hasTouch: true,
isMobile: true,
deviceScaleFactor: 1,
};
}],
},
and when I run this scraper, it will start with scraping page by page but the problem is that the scraper doesn't close craped page so every window is still open so after while actor reach crytical level of memory consuption and fall down.
Do you have any ideas how to fix it?
