verbal-limeV
Apify & Crawlee•2y ago•
4 replies
verbal-lime

Enqueue Links from new Window

Hi. I'm attempting to scrape data from a website, using crawlee and playwright, that is very JS heavy. The links I'm interested in are created by a JS function that opens the content in a new window. I've implemented the 
enqueueLinksByClickingElements
 function with a very specific selector. Playwright reports successfully clicking the links but I suspect the request is not being intercepted.
DEBUG Playwright Click Elements: enqueueLinksByClickingElements: There are 1 elements to click.
DEBUG Playwright Click Elements: enqueueLinksByClickingElements: Successfully clicked 1 elements out of 1
DEBUG PlaywrightCrawler: Crawled 1/2 pages, 0 failed requests.
DEBUG PlaywrightCrawler: Crawled 1/2 pages, 0 failed requests.
INFO  PlaywrightCrawler: All requests from the queue have been processed, the crawler will shut down.

I've also passed in the 
transformRequestFunction
 to set 
useExtendedUniqueKey
 to true. Is there a way I can:
1. Take a screenshot after Playwright clicks the element?
2. Log the intercepted requests?

Thanks!
Was this page helpful?