Can someone tell me what this error means? It says browser closed unexpectedly but not as to why.

2023-02-28T14:33:25.456Z ACTOR: Pulling Docker image from repository.
2023-02-28T14:33:26.293Z ACTOR: Creating Docker container.
2023-02-28T14:33:26.614Z ACTOR: Starting Docker container.
2023-02-28T14:34:04.313Z Traceback (most recent call last):
2023-02-28T14:34:04.314Z   File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
2023-02-28T14:34:04.315Z     return _run_code(code, main_globals, None,
2023-02-28T14:34:04.315Z   File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
2023-02-28T14:34:04.316Z     exec(code, run_globals)
2023-02-28T14:34:04.316Z   File "/usr/src/app/src/__main__.py", line 3, in <module>
2023-02-28T14:34:04.317Z     main()
2023-02-28T14:34:04.317Z   File "/usr/src/app/src/main.py", line 112, in main
2023-02-28T14:34:04.317Z     asyncio.get_event_loop().run_until_complete(run())
2023-02-28T14:34:04.318Z   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
2023-02-28T14:34:04.319Z     return future.result()
2023-02-28T14:34:04.319Z   File "/usr/src/app/src/main.py", line 30, in run
2023-02-28T14:34:04.320Z     browser = await launch(options=options)
2023-02-28T14:34:04.320Z   File "/opt/venv/lib/python3.9/site-packages/pyppeteer/launcher.py", line 307, in launch
2023-02-28T14:34:04.320Z     return await Launcher(options, **kwargs).launch()
2023-02-28T14:34:04.321Z   File "/opt/venv/lib/python3.9/site-packages/pyppeteer/launcher.py", line 168, in launch
2023-02-28T14:34:04.321Z     self.browserWSEndpoint = get_ws_endpoint(self.url)
2023-02-28T14:34:04.322Z   File "/opt/venv/lib/python3.9/site-packages/pyppeteer/launcher.py", line 227, in get_ws_endpoint
2023-02-28T14:34:04.322Z     raise BrowserError('Browser closed unexpectedly:\n')
2023-02-28T14:34:04.323Z pyppeteer.errors.BrowserError: Browser closed unexpectedly:

2023-02-28T14:33:25.456Z ACTOR: Pulling Docker image from repository.
2023-02-28T14:33:26.293Z ACTOR: Creating Docker container.
2023-02-28T14:33:26.614Z ACTOR: Starting Docker container.
2023-02-28T14:34:04.313Z Traceback (most recent call last):
2023-02-28T14:34:04.314Z   File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
2023-02-28T14:34:04.315Z     return _run_code(code, main_globals, None,
2023-02-28T14:34:04.315Z   File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
2023-02-28T14:34:04.316Z     exec(code, run_globals)
2023-02-28T14:34:04.316Z   File "/usr/src/app/src/__main__.py", line 3, in <module>
2023-02-28T14:34:04.317Z     main()
2023-02-28T14:34:04.317Z   File "/usr/src/app/src/main.py", line 112, in main
2023-02-28T14:34:04.317Z     asyncio.get_event_loop().run_until_complete(run())
2023-02-28T14:34:04.318Z   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
2023-02-28T14:34:04.319Z     return future.result()
2023-02-28T14:34:04.319Z   File "/usr/src/app/src/main.py", line 30, in run
2023-02-28T14:34:04.320Z     browser = await launch(options=options)
2023-02-28T14:34:04.320Z   File "/opt/venv/lib/python3.9/site-packages/pyppeteer/launcher.py", line 307, in launch
2023-02-28T14:34:04.320Z     return await Launcher(options, **kwargs).launch()
2023-02-28T14:34:04.321Z   File "/opt/venv/lib/python3.9/site-packages/pyppeteer/launcher.py", line 168, in launch
2023-02-28T14:34:04.321Z     self.browserWSEndpoint = get_ws_endpoint(self.url)
2023-02-28T14:34:04.322Z   File "/opt/venv/lib/python3.9/site-packages/pyppeteer/launcher.py", line 227, in get_ws_endpoint
2023-02-28T14:34:04.322Z     raise BrowserError('Browser closed unexpectedly:\n')
2023-02-28T14:34:04.323Z pyppeteer.errors.BrowserError: Browser closed unexpectedly:

2023-02-28T14:33:25.456Z ACTOR: Pulling Docker image from repository.
2023-02-28T14:33:26.293Z ACTOR: Creating Docker container.
2023-02-28T14:33:26.614Z ACTOR: Starting Docker container.
2023-02-28T14:34:04.313Z Traceback (most recent call last):
2023-02-28T14:34:04.314Z   File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
2023-02-28T14:34:04.315Z     return _run_code(code, main_globals, None,
2023-02-28T14:34:04.315Z   File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
2023-02-28T14:34:04.316Z     exec(code, run_globals)
2023-02-28T14:34:04.316Z   File "/usr/src/app/src/__main__.py", line 3, in <module>
2023-02-28T14:34:04.317Z     main()
2023-02-28T14:34:04.317Z   File "/usr/src/app/src/main.py", line 112, in main
2023-02-28T14:34:04.317Z     asyncio.get_event_loop().run_until_complete(run())
2023-02-28T14:34:04.318Z   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
2023-02-28T14:34:04.319Z     return future.result()
2023-02-28T14:34:04.319Z   File "/usr/src/app/src/main.py", line 30, in run
2023-02-28T14:34:04.320Z     browser = await launch(options=options)
2023-02-28T14:34:04.320Z   File "/opt/venv/lib/python3.9/site-packages/pyppeteer/launcher.py", line 307, in launch
2023-02-28T14:34:04.320Z     return await Launcher(options, **kwargs).launch()
2023-02-28T14:34:04.321Z   File "/opt/venv/lib/python3.9/site-packages/pyppeteer/launcher.py", line 168, in launch
2023-02-28T14:34:04.321Z     self.browserWSEndpoint = get_ws_endpoint(self.url)
2023-02-28T14:34:04.322Z   File "/opt/venv/lib/python3.9/site-packages/pyppeteer/launcher.py", line 227, in get_ws_endpoint
2023-02-28T14:34:04.322Z     raise BrowserError('Browser closed unexpectedly:\n')
2023-02-28T14:34:04.323Z pyppeteer.errors.BrowserError: Browser closed unexpectedly:

2023-02-28T14:33:25.456Z ACTOR: Pulling Docker image from repository.
2023-02-28T14:33:26.293Z ACTOR: Creating Docker container.
2023-02-28T14:33:26.614Z ACTOR: Starting Docker container.
2023-02-28T14:34:04.313Z Traceback (most recent call last):
2023-02-28T14:34:04.314Z   File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
2023-02-28T14:34:04.315Z     return _run_code(code, main_globals, None,
2023-02-28T14:34:04.315Z   File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
2023-02-28T14:34:04.316Z     exec(code, run_globals)
2023-02-28T14:34:04.316Z   File "/usr/src/app/src/__main__.py", line 3, in <module>
2023-02-28T14:34:04.317Z     main()
2023-02-28T14:34:04.317Z   File "/usr/src/app/src/main.py", line 112, in main
2023-02-28T14:34:04.317Z     asyncio.get_event_loop().run_until_complete(run())
2023-02-28T14:34:04.318Z   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
2023-02-28T14:34:04.319Z     return future.result()
2023-02-28T14:34:04.319Z   File "/usr/src/app/src/main.py", line 30, in run
2023-02-28T14:34:04.320Z     browser = await launch(options=options)
2023-02-28T14:34:04.320Z   File "/opt/venv/lib/python3.9/site-packages/pyppeteer/launcher.py", line 307, in launch
2023-02-28T14:34:04.320Z     return await Launcher(options, **kwargs).launch()
2023-02-28T14:34:04.321Z   File "/opt/venv/lib/python3.9/site-packages/pyppeteer/launcher.py", line 168, in launch
2023-02-28T14:34:04.321Z     self.browserWSEndpoint = get_ws_endpoint(self.url)
2023-02-28T14:34:04.322Z   File "/opt/venv/lib/python3.9/site-packages/pyppeteer/launcher.py", line 227, in get_ws_endpoint
2023-02-28T14:34:04.322Z     raise BrowserError('Browser closed unexpectedly:\n')
2023-02-28T14:34:04.323Z pyppeteer.errors.BrowserError: Browser closed unexpectedly:

rubber-blue•2/28/23, 2:51 PM

Are you running the code on Apify platform? Can you share the run id?

rubber-blue•2/28/23, 5:59 PM

Are you using the beta Python SDK template? There’s no browser in the default Docker image. So even if you install the pyppeteer, the Dockerfile is probably wrong and missing a lot of dependencies....

urgent-maroonOP•2/28/23, 6:50 PM

im using the regular python, not the beta sdk

urgent-maroonOP•2/28/23, 6:50 PM

here's the run id 6VKUYfzm58AaGWni5

rubber-blue•2/28/23, 8:13 PM

Thanks, we are checking it

rubber-blue•2/28/23, 8:16 PM

So yes - my answer above is correct. There's no browser in the image.

urgent-maroonOP•2/28/23, 8:35 PM

what's that mean? i can't use python for browser scraping?

urgent-maroonOP•2/28/23, 8:35 PM

or is there a way to add a browser to the image?

rubber-blue•2/28/23, 9:40 PM

You can, but you will have to take care of browser download and other necessary dependencies. The executable is just not bundled in this image.

urgent-maroonOP•2/28/23, 10:22 PM

the logs show it downloaded chromium i thought. is there something else i have to do for it?

rubber-blue•2/28/23, 10:32 PM

I don't think so - it downloaded pyppeteer, but not the browser executable: https://api.apify.com/v2/logs/ESecEJo1QfYsuhRBe. I haven't personally worked with custom docker images, so I would sadly won't be able to further advance you here, but it's missing the browser and maybe some other deps...

urgent-maroonOP•2/28/23, 10:37 PM

what is this then

2023-02-28T22:23:38.638Z [INFO] Starting Chromium download.
2023-02-28T22:23:39.154Z 
  0% 0.00/109M [00:00<?, ?b/s]
 21% 22.3M/109M [00:00<00:00, 223Mb/s]
 43% 46.8M/109M [00:00<00:00, 236Mb/s]
 65% 70.4M/109M [00:00<00:00, 236Mb/s]
 86% 94.1M/109M [00:00<00:00, 236Mb/s]
100% 109M/109M [00:00<00:00, 240Mb/s]
2023-02-28T22:23:39.155Z [INFO] Beginning extraction
2023-02-28T22:23:41.224Z [INFO] Chromium extracted to: /root/.local/share/pyppeteer/local-chromium/588429

2023-02-28T22:23:38.638Z [INFO] Starting Chromium download.
2023-02-28T22:23:39.154Z 
  0% 0.00/109M [00:00<?, ?b/s]
 21% 22.3M/109M [00:00<00:00, 223Mb/s]
 43% 46.8M/109M [00:00<00:00, 236Mb/s]
 65% 70.4M/109M [00:00<00:00, 236Mb/s]
 86% 94.1M/109M [00:00<00:00, 236Mb/s]
100% 109M/109M [00:00<00:00, 240Mb/s]
2023-02-28T22:23:39.155Z [INFO] Beginning extraction
2023-02-28T22:23:41.224Z [INFO] Chromium extracted to: /root/.local/share/pyppeteer/local-chromium/588429

rubber-blue•2/28/23, 10:44 PM

It's log from the run, not from the build though - I could guess that it's downloaded during the run, and pyppeteer is not really aware of it/not aware where it's extracted

urgent-maroonOP•2/28/23, 11:06 PM

oh, do you think adding an option that points to it would work?

rubber-blue•3/1/23, 10:57 AM

That might work, but that's still quite weird that in downloads during the run itself. Here's what I got from the team:

It’s most likely missing some dependencies and system libraries. The Node.js images have some complicated dependency installations to make sure the browser has everything it needs. When pyppeteer downloads Chromium, it does not install all the deps

It’s most likely missing some dependencies and system libraries. The Node.js images have some complicated dependency installations to make sure the browser has everything it needs. When pyppeteer downloads Chromium, it does not install all the deps

urgent-maroonOP•3/1/23, 2:58 PM

ah, that makes sense then. i'll try using actual puppeteer then, although i dont know any js

rubber-blue•3/1/23, 4:06 PM

Depending on the use-case - you could try our generic scrapers - https://apify.com/apify/web-scraper or https://apify.com/apify/playwright-scraper

I'm pretty sure you should pick it up - it does not requires much coding, mainly specifying the selectors, etc

Apify

Web Scraper · Apify

Crawls arbitrary websites using the Chrome browser and extracts data from pages using a provided JavaScript code. The actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.

Apify

Playwright Scraper · Apify

Crawls websites with the headless Chromium, Chrome, or Firefox browser and Playwright library using a provided server-side Node.js code. Supports both recursive crawling and a list of URLs. Supports login to a website.