isFinishedFunction, check other crawler?

Hello,

two questions.

Is there a way to call this.isFinishedFunction so it calls the original function but also just add another web_crawler_queue isFinished on top of it? the uncommented out function I tried, worked somewhat, but, after a long running web_crawler_queue finished, it just kept giving me a stalled error, and, this crawler that this function belongs to never finished.

    autoscaledPoolOptions: {
      isFinishedFunction: async () => {
        const web_crawler_queue = await RequestQueue.open(place_id)
//        return this.isFinishedFunction() && await web_crawler_queue.isFinished()
        return await request_queue.isFinished() && await web_crawler_queue.isFinished()

      }
    },

    autoscaledPoolOptions: {
      isFinishedFunction: async () => {
        const web_crawler_queue = await RequestQueue.open(place_id)
//        return this.isFinishedFunction() && await web_crawler_queue.isFinished()
        return await request_queue.isFinished() && await web_crawler_queue.isFinished()

      }
    },

MEE6•12/16/23, 12:37 AM

@bmax just advanced to level 3! Thanks for your contributions!

Lukas Krivka•12/19/23, 11:49 AM

There is no original function since you are putting your own there in options. If you would want to use original, you would have to monkeypatch it after defining.

technological-jadeOP•12/20/23, 9:55 PM

got it, will try this

technological-jadeOP•12/20/23, 9:55 PM

my code above almost works @Lukas Krivka but for some reason doesn't all the time..

technological-jadeOP•12/20/23, 10:01 PM

which is basically is my current request_queue finished and is my other crawler finished

technological-jadeOP•12/20/23, 10:01 PM

but sometimes it doesn't pick up and I'll get an log along the lines of : "x, y, z ahs been stalled for 350 seconds"

technological-jadeOP•12/21/23, 2:44 AM

technological-jadeOP•12/21/23, 3:06 AM

this: