2 replies

How to push request queue after click method?

I tried to push a link in the request queue through repeat.
However, when I looked at the contents of the queue, the queue count did not change or the expected count did not come out.
I think it is duplicated as the crawl proceeds at the same time, and the same link url enters the queue.
Is there a solution?

for (let i = 0; i < 5; i++) {
    await enqueueLinks({
      selector: "li.sa_item>div>div>div.sa_text>a",
      label: "DETAIL",
      strategy: EnqueueStrategy.All,
    })
    await page.waitForSelector('div.section_more');
    await page.click('div.section_more');
    await page.waitForLoadState("domcontentloaded");
    const queueInfo = await crawler.requestQueue?.getInfo();
    console.log(queueInfo);
  }

for (let i = 0; i < 5; i++) {
    await enqueueLinks({
      selector: "li.sa_item>div>div>div.sa_text>a",
      label: "DETAIL",
      strategy: EnqueueStrategy.All,
    })
    await page.waitForSelector('div.section_more');
    await page.click('div.section_more');
    await page.waitForLoadState("domcontentloaded");
    const queueInfo = await crawler.requestQueue?.getInfo();
    console.log(queueInfo);
  }

How to push request queue after click method?

Similar Threads