eastern-cyan
eastern-cyan•2y ago

How to use the enqueueLinksByClickingElements function?

I'm trying to grab the link from each card by clicking them (just like you would with enqueueLinks) but with this function it's not working. I don't know what am I doing wrong here. 
await enqueueLinksByClickingElements({
    selector: 'p.prod__picture',
    label: 'PRODUCT_DETAILS'
  });
await enqueueLinksByClickingElements({
    selector: 'p.prod__picture',
    label: 'PRODUCT_DETAILS'
  });
Right now, the function clicks on the first p.prod__picture element, loads its detail page and then the crawler closes.
No description
5 Replies
Lukas Krivka
Lukas Krivka•2y ago
There is probably some interactivity on the page that prevents further clicking. You would need to share the website
eastern-cyan
eastern-cyanOP•2y ago
rare-sapphire
rare-sapphire•2y ago
the label here you must mention the request handler label that you want the links handled in
eastern-cyan
eastern-cyanOP•2y ago
I have settled up this handler: 
router.addHandler('PRODUCT_DETAILS', async ({ page, request, log }) => {
  log.info(`PRODUCT_DETAILS ${request.url}`);

  const prodName = await page.locator('p.prod__name').textContent();
  const prodPrice = await page.$eval('.cart__actions__price > .base__price', (price) =>
    price.textContent.replaceAll(/\$|\s|\./g, '')
  );

  console.log({ prodName, prodPrice });
});
router.addHandler('PRODUCT_DETAILS', async ({ page, request, log }) => {
  log.info(`PRODUCT_DETAILS ${request.url}`);

  const prodName = await page.locator('p.prod__name').textContent();
  const prodPrice = await page.$eval('.cart__actions__price > .base__price', (price) =>
    price.textContent.replaceAll(/\$|\s|\./g, '')
  );

  console.log({ prodName, prodPrice });
});
and this is the result in the console: 
INFO  PlaywrightCrawler: Starting the crawler.
INFO  PlaywrightCrawler: PRODUCT_DETAILS https://domicilios.tiendasd1.com/p/tomate-chonto-x-1000-g-12005556
{ prodName: 'TOMATE CHONTO X 1000 G', prodPrice: '4190' }
INFO  PlaywrightCrawler: All requests from the queue have been processed, the crawler will shut down.
INFO  PlaywrightCrawler: Final request statistics: {"requestsFinished":2,"requestsFailed":0,"retryHistogram":[2],"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":8255,"requestsFinishedPerMinute":7,"requestsFailedPerMinute":0,"requestTotalDurationMillis":16509,"requestsTotal":2,"crawlerRuntimeMillis":16632}
INFO  PlaywrightCrawler: Finished! Total 2 requests: 2 succeeded, 0 failed. {"terminal":true}
INFO  PlaywrightCrawler: Starting the crawler.
INFO  PlaywrightCrawler: PRODUCT_DETAILS https://domicilios.tiendasd1.com/p/tomate-chonto-x-1000-g-12005556
{ prodName: 'TOMATE CHONTO X 1000 G', prodPrice: '4190' }
INFO  PlaywrightCrawler: All requests from the queue have been processed, the crawler will shut down.
INFO  PlaywrightCrawler: Final request statistics: {"requestsFinished":2,"requestsFailed":0,"retryHistogram":[2],"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":8255,"requestsFinishedPerMinute":7,"requestsFailedPerMinute":0,"requestTotalDurationMillis":16509,"requestsTotal":2,"crawlerRuntimeMillis":16632}
INFO  PlaywrightCrawler: Finished! Total 2 requests: 2 succeeded, 0 failed. {"terminal":true}
MEE6
MEE6•2y ago
@Dylan just advanced to level 1! Thanks for your contributions! 🎉

Did you find this page helpful?