Apify Discord Mirror

Updated last year

dynamic rendering with button clicking

At a glance

The community member in the post is asking how to keep clicking and waiting a second, and clicking a button that renders more data in a Playwright crawler, until the button is no longer present, while also scraping the rendered product URLs. A community member in the comments suggests a simple for loop with a sleep, where the loop checks for the presence of the button, clicks it if it exists, and breaks the loop if the button is not found for a certain number of times.

hharish

in a playwright crawler, how could i keep clicking and waiting a second and clicking a button that renders in more data until that button is no longer present and end the loop

and while the clicking is going on, rendered product urls are scraped

1 comment

LLukas Krivka

I think you can do a simple for loop with a sleep. something like:

Plain Text

let noMoreButtonTimes = 0;
for (;;) {
    const button = await page.$('button')
    if (button) {
        await button.click()
    } else {
       noMoreButtonTimes++;
       if (noMoreButtonTimes > 3) break 
    }
    await sleep(2000)
}

Add a reply