sensitive-blue•3y ago
scraping different website strutures
can crawlee be used for automating scraping of diff website structures
can you make a code to scrape a website structure without having to spend time inspecting each page for specific html elements for retrieving the data you need
4 Replies
This is called generic scraping and you can do it for things that are the same on each website, you can make regexes for phone numbers, collect metadata etc. but thing like product data are really hard and advanced AI is needed.
jolly-crimson•3y ago
Do you have an example where I could find or learn more about advanced AI?
@GoranTopic just advanced to level 2! Thanks for your contributions! 🎉
@GoranTopic Those are highly specialized tools that you can google with like
product AI scraper