Lliving-lavender
retireBrowserAfterPageCount does not work with values above close to 30 for Playwright
Hi,
I am using Crawlee with Playwright. Some Pages require a login procedure, which works well by simply using page actions in Playwright. I am running into a problem though, because Crawlee opens a new browser after around every 30 pages, which deletes all Cookies/Local Storage and therefore fails the login. Relogging can be difficult or sometimes not technically possible.
I tried using the browserPoolOptions to prevent the launch of a new browser like this:
browserPoolOptions: {
retireBrowserAfterPageCount: 50,
},
Setting the value to 1 works well, but setting it to 50 or higher numbers does not have any effect. It still launches a new browser after around 30 pages.
Is this a known feature/bug? It would help greatly if I could run a crawl on one browser instance for several hundred pages, even if it is not recommended.
Thank you for your help!
I am using Crawlee with Playwright. Some Pages require a login procedure, which works well by simply using page actions in Playwright. I am running into a problem though, because Crawlee opens a new browser after around every 30 pages, which deletes all Cookies/Local Storage and therefore fails the login. Relogging can be difficult or sometimes not technically possible.
I tried using the browserPoolOptions to prevent the launch of a new browser like this:
browserPoolOptions: {
retireBrowserAfterPageCount: 50,
},
Setting the value to 1 works well, but setting it to 50 or higher numbers does not have any effect. It still launches a new browser after around 30 pages.
Is this a known feature/bug? It would help greatly if I could run a crawl on one browser instance for several hundred pages, even if it is not recommended.
Thank you for your help!
This is the official developer community of Apify and Crawlee.
14,091Members
Resources
Similar Threads
Was this page helpful?
