New page middleware

I need to use adblock from here https://www.npmjs.com/package/@cliqz/adblocker-playwright, but not quite sure what is the best way to integrate it with Crawlee. For now, I create page with dummy url, enable adblocker on the page and then navigate to the target website. I am curious is there an API to add some kind of middleware to the page creation process, to add adblocker to every newly created page before target url navigation? Couldn't find anything suitable in neither Crawlee or Playwright docs

Thank you in advance!

Lukas Krivka•9/30/22, 12:52 PM

You can run this function is preNavigationHookspreNavigationHooks where you have access to the pagepage.

PlaywrightBlocker.fromPrebuiltAdsAndTracking(fetch).then((blocker) => {
  blocker.enableBlockingInPage(page);
});

PlaywrightBlocker.fromPrebuiltAdsAndTracking(fetch).then((blocker) => {
  blocker.enableBlockingInPage(page);
});

rubber-blueOP•9/30/22, 1:21 PM

Thank you very much!

Alexey Udovydchenko•9/30/22, 8:58 PM

docs: https://crawlee.dev/api/puppeteer-crawler/interface/PuppeteerCrawlerOptions#preNavigationHooks

pls be aware it will be called on retry as well