Apify Discord Mirror

Updated 5 months ago

Private web scraping?

At a glance

The community member is new to Apify and wants to know if it can be used to scrape websites that are not publicly accessible, such as internal corporate knowledge bases. They ask if there is a way to use a proxy or other method to connect Apify to these private websites. One comment indicates that Apify can only be used to scrape public websites, but there is no explicitly marked answer to the community member's question.

Hello Friends! I'm new to Apify and pretty excited about what I've learned so far. One use case I'm not sure of: Can Apify be used to scrape a website that's not on the public internet? Specifically, I want to scrape knowledgebases inside corporations (with their permission). Is there for example some sort of proxy that could be put in place inside the private network that connects with Appify and then scrapes at Apify's direction? Or etc? (I've cross-posted this in the Crawlee help forum; hopefully that's ok)
L
1 comment
No, only sites on public
Add a reply
Sign up and join the conversation on Discord