8 replies

How to configure PlaywrightCrawler or PuppeteerCrawler with Browserless?

🎭PlaywrightCrawler🖥️PuppeteerCrawler👨‍💻Web-Scraping💭Suggestions

Hello, I have no success in setting up either of the crawlers with Browserless.

What is the proper Typescript way to do that with Browserless? Any docs? Thank you!

⁨⁨

const crawler = new PuppeteerCrawler({
    launchContext: {
        useChrome: true,
        launcher: puppeteer,
        launchOptions: {
          browserWSEndpoint: wsEndpoint, => not even connected with this option
          args: [
            '--no-sandbox',
            '--disable-setuid-sandbox',
            '--disable-dev-shm-usage'
          ]
        }
      },

const crawler = new PuppeteerCrawler({
    launchContext: {
        useChrome: true,
        launcher: puppeteer,
        launchOptions: {
          browserWSEndpoint: wsEndpoint, => not even connected with this option
          args: [
            '--no-sandbox',
            '--disable-setuid-sandbox',
            '--disable-dev-shm-usage'
          ]
        }
      },

⁩⁩

⁨⁨

const crawler = new PlaywrightCrawler({
    launchContext: {
        launchOptions: {
            browserWSEndpoint: wsEndpoint, => not even has this property
        }, 
    },

const crawler = new PlaywrightCrawler({
    launchContext: {
        launchOptions: {
            browserWSEndpoint: wsEndpoint, => not even has this property
        }, 
    },

⁩⁩

Solution

If you want to use a crawler that just handles HTTP requests you should use the Cheerio Crawler https://crawlee.dev/js/docs/guides/cheerio-crawler-guide

CheerioCrawler guide | Crawlee for JavaScript · Build reliable cra...

Crawlee helps you build and maintain your crawlers. It's open source, but built by developers who scrape millions of pages every day for a living.

Jump to solution

Apify & Crawlee•4mo ago•

8 replies

Wei

How to configure PlaywrightCrawler or PuppeteerCrawler with Browserless?

🎭PlaywrightCrawler🖥️PuppeteerCrawler👨‍💻Web-Scraping💭Suggestions

Hello, I have no success in setting up either of the crawlers with Browserless.

What is the proper Typescript way to do that with Browserless? Any docs? Thank you!

⁨⁨

const crawler = new PuppeteerCrawler({
    launchContext: {
        useChrome: true,
        launcher: puppeteer,
        launchOptions: {
          browserWSEndpoint: wsEndpoint, => not even connected with this option
          args: [
            '--no-sandbox',
            '--disable-setuid-sandbox',
            '--disable-dev-shm-usage'
          ]
        }
      },

const crawler = new PuppeteerCrawler({
    launchContext: {
        useChrome: true,
        launcher: puppeteer,
        launchOptions: {
          browserWSEndpoint: wsEndpoint, => not even connected with this option
          args: [
            '--no-sandbox',
            '--disable-setuid-sandbox',
            '--disable-dev-shm-usage'
          ]
        }
      },

⁩⁩

⁨⁨

const crawler = new PlaywrightCrawler({
    launchContext: {
        launchOptions: {
            browserWSEndpoint: wsEndpoint, => not even has this property
        }, 
    },

const crawler = new PlaywrightCrawler({
    launchContext: {
        launchOptions: {
            browserWSEndpoint: wsEndpoint, => not even has this property
        }, 
    },

⁩⁩

Solution

If you want to use a crawler that just handles HTTP requests you should use the Cheerio Crawler https://crawlee.dev/js/docs/guides/cheerio-crawler-guide

CheerioCrawler guide | Crawlee for JavaScript · Build reliable cra...

Crawlee helps you build and maintain your crawlers. It's open source, but built by developers who scrape millions of pages every day for a living.

Jump to solution

Continue the conversation

Join the Discord to ask follow-up questions and connect with the community

A&C

Apify & Crawlee

This is the official developer community of Apify and Crawlee.

14,851 Members

Join

How to configure PlaywrightCrawler or PuppeteerCrawler with Browserless?

How to configure PlaywrightCrawler or PuppeteerCrawler with Browserless?

Continue the conversation

Apify & Crawlee

Continue the conversation

Apify & Crawlee

Similar Threads