Override browser permission on PuppeteerCrawler

At a glance

A community member is having an issue overriding permissions when using the PuppeteerCrawler. They provided some code to try and override the permissions, but are getting a warning message about not being able to read a private member. The community members discuss the issue, with one suggesting to try getting the context via page.browserContext() instead. This solution works, and the community member thanks the others for their help.

mmicroworlds

Hi, a quick question: how do I override a certain permission on a page when using PuppeteerCrawler? Something like so:

Plain Text

...,
preNavigationHooks: [
  async (crawlingContext, gotoOptions) => {
    const { page, browserController: { browser } } = crawlingContext;

    const context = await browser.defaultBrowserContext();
    context.clearPermissionOverrides();
    context.overridePermissions('some-domain', ['some-feature']);
  }
],
...

When I run it, I get a warning message:

WARN  PuppeteerCrawler: Reclaiming failed request back to the list or queue. Cannot read private member from an object whose class did not declare it

I understand that I'm accessing private prop but how/where do I override such permissions on PuppeteerCrawler? 🙂

Thanks! 🙏

11 comments

mmicroworlds

Hi , can you please assist with this? 🙏

Oh..interesting

Let me take a look

Thank you!

do you happen to have a stacktrace too?

vvladdy

also, can you try getting the context via page.browserContext() instead (followed by the same context.overridePermissions calls)?

mmicroworlds

@stacktrace:

Plain Text

ERROR PuppeteerCrawler: Request failed and reached maximum retries. TypeError: Cannot read private member from an object whose class did not declare it
    at __classPrivateFieldGet (/path/node_modules/puppeteer/lib/cjs/puppeteer/common/Browser.js:25:94)
    at Proxy.defaultBrowserContext (/path/node_modules/puppeteer/lib/cjs/puppeteer/common/Browser.js:213:16)
    at PuppeteerCrawler.preNavigationHooks (file:///path/src/index.js:46:39)
    at PuppeteerCrawler._executeHooks (/path/node_modules/@crawlee/basic/internals/basic-crawler.js:794:23)
    at PuppeteerCrawler._handleNavigation (/path/node_modules/@crawlee/browser/internals/browser-crawler.js:263:20)
    at PuppeteerCrawler._runRequestHandler (/path/node_modules/@crawlee/browser/internals/browser-crawler.js:215:28)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async PuppeteerCrawler._runRequestHandler (/path/node_modules/@crawlee/puppeteer/internals/puppeteer-crawler.js:102:9)
    at async wrap (/path/node_modules/@apify/timeout/index.js:52:21)

mmicroworlds

Cool, testing...

mmicroworlds

Perfect! That works. Thanks! ✨

vvladdy

oh, sweet

vvladdy

No worries

Add a reply

Apify Discord Mirror

Override browser permission on PuppeteerCrawler