Only the first crawler runs in function

When running the example below, only the first crawler (crawler1) runs, and the second crawler (crawler2) does not work as intended. Running either crawler individually works fine, and changing the URL to something completely different also works fine. Here is an example.

import { PlaywrightCrawler } from 'crawlee';

export async function runExample() {
  const testPage1 =
    'https://inspections.healthunit.com/HedgehogPortal/#/18fbee00-f0a3-49e3-b323-9153b6c4924c/disclosure/facility/3448568d-737b-4b41-ab63-1f2d7a2252b5';

  const testPage2 =
    'https://inspections.healthunit.com/HedgehogPortal/#/18fbee00-f0a3-49e3-b323-9153b6c4924c/disclosure/facility/3448568d-737b-4b41-ab63-1f2d7a2252b5/inspection/ac3196c5-13e6-486c-8b9c-b85dd019fc05';

  const crawler1 = new PlaywrightCrawler({
    requestHandler: async ({ request, page, log }) => {
      const title = await page.title();
      log.info(`URL: ${request.url}\nTITLE: ${title}`);
    },
    launchContext: {
      launchOptions: {
        args: ['--ignore-certificate-errors'],
      },
    },
  });

  const crawler2 = new PlaywrightCrawler({
    requestHandler: async ({ request, page, log }) => {
      const title = await page.title();
      log.info(`URL: ${request.url}\nTITLE: ${title}`);
    },
    launchContext: {
      launchOptions: {
        args: ['--ignore-certificate-errors'],
      },
    },
  });

  await crawler1.run([testPage1]);
  await crawler2.run([testPage2]);
}

runExample();

import { PlaywrightCrawler } from 'crawlee';

export async function runExample() {
  const testPage1 =
    'https://inspections.healthunit.com/HedgehogPortal/#/18fbee00-f0a3-49e3-b323-9153b6c4924c/disclosure/facility/3448568d-737b-4b41-ab63-1f2d7a2252b5';

  const testPage2 =
    'https://inspections.healthunit.com/HedgehogPortal/#/18fbee00-f0a3-49e3-b323-9153b6c4924c/disclosure/facility/3448568d-737b-4b41-ab63-1f2d7a2252b5/inspection/ac3196c5-13e6-486c-8b9c-b85dd019fc05';

  const crawler1 = new PlaywrightCrawler({
    requestHandler: async ({ request, page, log }) => {
      const title = await page.title();
      log.info(`URL: ${request.url}\nTITLE: ${title}`);
    },
    launchContext: {
      launchOptions: {
        args: ['--ignore-certificate-errors'],
      },
    },
  });

  const crawler2 = new PlaywrightCrawler({
    requestHandler: async ({ request, page, log }) => {
      const title = await page.title();
      log.info(`URL: ${request.url}\nTITLE: ${title}`);
    },
    launchContext: {
      launchOptions: {
        args: ['--ignore-certificate-errors'],
      },
    },
  });

  await crawler1.run([testPage1]);
  await crawler2.run([testPage2]);
}

runExample();

Apify & Crawlee•14mo ago•

10 replies

wispy-olive

Only the first crawler runs in function

import { PlaywrightCrawler } from 'crawlee';

export async function runExample() {
  const testPage1 =
    'https://inspections.healthunit.com/HedgehogPortal/#/18fbee00-f0a3-49e3-b323-9153b6c4924c/disclosure/facility/3448568d-737b-4b41-ab63-1f2d7a2252b5';

  const testPage2 =
    'https://inspections.healthunit.com/HedgehogPortal/#/18fbee00-f0a3-49e3-b323-9153b6c4924c/disclosure/facility/3448568d-737b-4b41-ab63-1f2d7a2252b5/inspection/ac3196c5-13e6-486c-8b9c-b85dd019fc05';

  const crawler1 = new PlaywrightCrawler({
    requestHandler: async ({ request, page, log }) => {
      const title = await page.title();
      log.info(`URL: ${request.url}\nTITLE: ${title}`);
    },
    launchContext: {
      launchOptions: {
        args: ['--ignore-certificate-errors'],
      },
    },
  });

  const crawler2 = new PlaywrightCrawler({
    requestHandler: async ({ request, page, log }) => {
      const title = await page.title();
      log.info(`URL: ${request.url}\nTITLE: ${title}`);
    },
    launchContext: {
      launchOptions: {
        args: ['--ignore-certificate-errors'],
      },
    },
  });

  await crawler1.run([testPage1]);
  await crawler2.run([testPage2]);
}

runExample();

import { PlaywrightCrawler } from 'crawlee';

export async function runExample() {
  const testPage1 =
    'https://inspections.healthunit.com/HedgehogPortal/#/18fbee00-f0a3-49e3-b323-9153b6c4924c/disclosure/facility/3448568d-737b-4b41-ab63-1f2d7a2252b5';

  const testPage2 =
    'https://inspections.healthunit.com/HedgehogPortal/#/18fbee00-f0a3-49e3-b323-9153b6c4924c/disclosure/facility/3448568d-737b-4b41-ab63-1f2d7a2252b5/inspection/ac3196c5-13e6-486c-8b9c-b85dd019fc05';

  const crawler1 = new PlaywrightCrawler({
    requestHandler: async ({ request, page, log }) => {
      const title = await page.title();
      log.info(`URL: ${request.url}\nTITLE: ${title}`);
    },
    launchContext: {
      launchOptions: {
        args: ['--ignore-certificate-errors'],
      },
    },
  });

  const crawler2 = new PlaywrightCrawler({
    requestHandler: async ({ request, page, log }) => {
      const title = await page.title();
      log.info(`URL: ${request.url}\nTITLE: ${title}`);
    },
    launchContext: {
      launchOptions: {
        args: ['--ignore-certificate-errors'],
      },
    },
  });

  await crawler1.run([testPage1]);
  await crawler2.run([testPage2]);
}

runExample();

Only the first crawler runs in function

Similar Threads

Only the first crawler runs in function

Similar Threads

Similar Threads

Similar Threads