crawler process not exiting after teardown is called.

I am running a Playwright crawler and I call await crawler.teardown()await crawler.teardown() after some condition is met,

Issue with this approach is that the crawlee process hangs indefinitely and doesn't exit even after a while. I have to force stop it using CTRL + C, on searching github, it looks like its an issue with some timeouts but I havent set one anywhere.

here's how I am initializing my crawler:

export const crawler = new PlaywrightCrawler({
    requestHandler:router,
    maxRequestsPerMinute:10,
})

export const crawler = new PlaywrightCrawler({
    requestHandler:router,
    maxRequestsPerMinute:10,
})

and I have a playwright config where I have set the default timeout

import { defineConfig } from '@playwright/test';

export default defineConfig({
    timeout: 3000,
});

import { defineConfig } from '@playwright/test';

export default defineConfig({
    timeout: 3000,
});

Apify & Crawlee•3y ago•

3 replies

verbal-lime

crawler process not exiting after teardown is called.

export const crawler = new PlaywrightCrawler({
    requestHandler:router,
    maxRequestsPerMinute:10,
})

export const crawler = new PlaywrightCrawler({
    requestHandler:router,
    maxRequestsPerMinute:10,
})

and I have a playwright config where I have set the default timeout

import { defineConfig } from '@playwright/test';

export default defineConfig({
    timeout: 3000,
});

import { defineConfig } from '@playwright/test';

export default defineConfig({
    timeout: 3000,
});

Continue the conversation

Join the Discord to ask follow-up questions and connect with the community

A&C

Apify & Crawlee

This is the official developer community of Apify and Crawlee.

14,851 Members

Join

crawler process not exiting after teardown is called.

crawler process not exiting after teardown is called.

Continue the conversation

Apify & Crawlee

Continue the conversation

Apify & Crawlee

Similar Threads