old-apricot•2y ago

How to quit PuppeteerCrawler automatically

Hey folks, I’ve been working on this crawler using PuppeteerCrawler in Javascript, all set up in VSCode on my Windows machine. I’ve got maxRequestsPerCrawl set up and all, but for some reason, it’s not going back to waiting for input in the terminal. I literally have to hit Ctrl+C to kill the app that I started with npm start before it’ll wait for input again. Anyone got any ideas on how to get it to shut down on its own?

4 Replies

metropolitan-bronze•2y ago

It's hard to assist without code snippets. But you can create some kind of condition to exit the actor like so:

if (condition === satisfied) {
  await Actor.exit();
}

if (condition === satisfied) {
  await Actor.exit();
}

Lukas Krivka•2y ago

How do you start the process? Might be a bug specifically with Windows. Also share the code pls

old-apricotOP•2y ago

@microworlds Hey there, I’ve been messing around with this bit of code:

await crawler.run(startUrls);
Actor.exit();

await crawler.run(startUrls);
Actor.exit();

But, it’s been giving me a headache because it keeps throwing errors. 🤷‍♂️ @Lukas Krivka I used to get the ball rolling with npm start, swapped out Actor.exit(); for process.exit();, and then when I fired it up with node src/main.js right in the project folder, it shut down like a charm. Cheers for the advice! 👍

Lukas Krivka•2y ago

That indeed suggests you had some hanging promise before. The Node.js process will not finish until all promises are done or you have to kill it explicitly like process.exit() or await Actor.exit() (this is Apify SDK only).

How to quit PuppeteerCrawler automatically

Did you find this page helpful?