Ddeveloped-peach
Web Page Finder
New to Apify. Need advice on best Actor to complete desired outcome.
Goal: Find all websites using a set of specific root urls.
Use case: A known company supplies it clients with dedicated webpages using a root url that doesn't itself display information. To get to a page with information, you need to have a complete url which always follows this format:
http://company-url.com/client-identifier
Where company-url is the creator/host of the webpages and client-identifier is the client name or some version of it.
Usually we can manually put in google search "inurl:http://company-url.com/"*" and manually find each result.
Can this be automated in Apify? Whic Actor would complete this? What settings do I need to be aware of for optimal results?
Thanks!
Goal: Find all websites using a set of specific root urls.
Use case: A known company supplies it clients with dedicated webpages using a root url that doesn't itself display information. To get to a page with information, you need to have a complete url which always follows this format:
http://company-url.com/client-identifier
Where company-url is the creator/host of the webpages and client-identifier is the client name or some version of it.
Usually we can manually put in google search "inurl:http://company-url.com/"*" and manually find each result.
Can this be automated in Apify? Whic Actor would complete this? What settings do I need to be aware of for optimal results?
Thanks!
This is the official developer community of Apify and Crawlee.
14,091Members
Resources
Recent Announcements
Similar Threads
Was this page helpful?
