8 replies

How to configure PlaywrightCrawler or PuppeteerCrawler with Browserless?

🎭PlaywrightCrawler🖥️PuppeteerCrawler👨‍💻Web-Scraping💭Suggestions

Hello, I have no success in setting up either of the crawlers with Browserless.

What is the proper Typescript way to do that with Browserless? Any docs? Thank you!

⁨⁨

const crawler = new PuppeteerCrawler({
    launchContext: {
        useChrome: true,
        launcher: puppeteer,
        launchOptions: {
          browserWSEndpoint: wsEndpoint, => not even connected with this option
          args: [
            '--no-sandbox',
            '--disable-setuid-sandbox',
            '--disable-dev-shm-usage'
          ]
        }
      },

const crawler = new PuppeteerCrawler({
    launchContext: {
        useChrome: true,
        launcher: puppeteer,
        launchOptions: {
          browserWSEndpoint: wsEndpoint, => not even connected with this option
          args: [
            '--no-sandbox',
            '--disable-setuid-sandbox',
            '--disable-dev-shm-usage'
          ]
        }
      },

⁩⁩

⁨⁨

const crawler = new PlaywrightCrawler({
    launchContext: {
        launchOptions: {
            browserWSEndpoint: wsEndpoint, => not even has this property
        }, 
    },

const crawler = new PlaywrightCrawler({
    launchContext: {
        launchOptions: {
            browserWSEndpoint: wsEndpoint, => not even has this property
        }, 
    },

⁩⁩

Solution

If you want to use a crawler that just handles HTTP requests you should use the Cheerio Crawler https://crawlee.dev/js/docs/guides/cheerio-crawler-guide

CheerioCrawler guide | Crawlee for JavaScript · Build reliable cra...

Crawlee helps you build and maintain your crawlers. It's open source, but built by developers who scrape millions of pages every day for a living.

Jump to solution

Apify & Crawlee•2mo ago•

8 replies

Wei

How to configure PlaywrightCrawler or PuppeteerCrawler with Browserless?

🎭PlaywrightCrawler🖥️PuppeteerCrawler👨‍💻Web-Scraping💭Suggestions

Hello, I have no success in setting up either of the crawlers with Browserless.

What is the proper Typescript way to do that with Browserless? Any docs? Thank you!

⁨⁨

const crawler = new PuppeteerCrawler({
    launchContext: {
        useChrome: true,
        launcher: puppeteer,
        launchOptions: {
          browserWSEndpoint: wsEndpoint, => not even connected with this option
          args: [
            '--no-sandbox',
            '--disable-setuid-sandbox',
            '--disable-dev-shm-usage'
          ]
        }
      },

const crawler = new PuppeteerCrawler({
    launchContext: {
        useChrome: true,
        launcher: puppeteer,
        launchOptions: {
          browserWSEndpoint: wsEndpoint, => not even connected with this option
          args: [
            '--no-sandbox',
            '--disable-setuid-sandbox',
            '--disable-dev-shm-usage'
          ]
        }
      },

⁩⁩

⁨⁨

const crawler = new PlaywrightCrawler({
    launchContext: {
        launchOptions: {
            browserWSEndpoint: wsEndpoint, => not even has this property
        }, 
    },

const crawler = new PlaywrightCrawler({
    launchContext: {
        launchOptions: {
            browserWSEndpoint: wsEndpoint, => not even has this property
        }, 
    },

⁩⁩

Solution

If you want to use a crawler that just handles HTTP requests you should use the Cheerio Crawler https://crawlee.dev/js/docs/guides/cheerio-crawler-guide

CheerioCrawler guide | Crawlee for JavaScript · Build reliable cra...

Crawlee helps you build and maintain your crawlers. It's open source, but built by developers who scrape millions of pages every day for a living.

Jump to solution

How to configure PlaywrightCrawler or PuppeteerCrawler with Browserless?

Similar Threads

How to configure PlaywrightCrawler or PuppeteerCrawler with Browserless?

Similar Threads

Similar Threads

Similar Threads