How can you find HTML element that can be clicked on to...

At a glance

The community member is having trouble finding the HTML element for a specific dropdown on a webpage. They ask if there is any HTML attribute they can look at to determine which elements can be clicked. Another community member suggests using the page.selectOption() method from Playwright, which can select options from dropdowns even if they are not initially visible. The community member also provides a link to the webpage and specifies that they want to click on the "FULL" dropdown.

Another community member suggests that the title element on the page must be clicked first in order for the dropdown to be added to the DOM. They provide sample code that clicks on all the title elements, then selects the "NONE" option from all the "FULL" dropdowns that are now visible.

There is no explicitly marked answer, but the community members provide helpful suggestions and sample code to solve the issue.

Useful resources

CCasper

I have managed to find element for some dropdowns but for a specific dropdown I cant find it.
Is there any HTML attribute I can look at to determine which can be clicked? There does not seem to be a onClick() or similar

12 comments

tthek1tten

Got a link to the page and a screenshot of what you want to click?

CCasper

Attachment

CCasper

https://www.europarl.europa.eu/RegistreWeb/search/simpleSearchHome.htm?languages=EN&sortAndOrder=DATE_DOCU_DESC

CCasper

I want to click on "FULL"

CCasper

dropdown

CCasper

Forgot to mention you need to click on the top element on the page before it is shown as displayed

tthek1tten

There is page.selectOption(): https://playwright.dev/docs/api/class-page#page-select-option

This selector seems to match all the dropdowns on the page that you want:

Plain Text

select[name*="FULL"]

Since all the select elements are already PRESENT in the DOM (just not visible until the container is clicked), you can do:

Plain Text

page.selectOption('selector-here', 'NONE', { force: true });

force set to true means the element doesn't necessarily have to be visible (see more here: https://playwright.dev/docs/actionability)

But since you want to click all of them, you'll need to loop through all elements that that selector resolves to and run the selecting logic for each one.

tthek1tten

Hope this helps 😄

CCasper

Thanks I will try that out. Propably better than just selecting each element and trying to click it 😛

CCasper

How would the corresponding code look like in chrome dev tools?

tthek1tten

So I was wrong about the page. The title MUST be clicked for the <select> to be added to the DOM. Try running this code:

Plain Text

import { PlaywrightCrawler } from 'crawlee';

const crawler = new PlaywrightCrawler({
    headless: false,
    requestHandler: async ({ page }) => {
        // Wait for the requests for the data to be made
        await page.waitForLoadState('networkidle');

        const titles = await page.$$('h3[class*="results-item-title"]');

        // Click all title elements to display the dropdown on the page
        for (const title of titles) {
            await title.click();
        }

        const dropdowns = await page.$$('select[name*="FULL"]');

        // Choose the NONE option on all dropdowns
        for (const dropdown of dropdowns) {
            await dropdown.selectOption('NONE');
        }

        // give 20 seconds to check out the page
        await page.waitForTimeout(20e3);
    },
});

await crawler.run(['https://www.europarl.europa.eu/RegistreWeb/search/simpleSearchHome.htm?languages=EN&sortAndOrder=DATE_DOCU_DESC']);

CCasper

thanks I will test it soon

Add a reply

Apify Discord Mirror

How can you find HTML element that can be clicked on to use dropdown?