UWeb Site Content Monitor Tool to scrape new pages
Hello everyone,
I mostly use website content crawler on apify store for adding the content into pinecone which is my vector database for passing these vector into langchain LLM to create chatbot using Python. I wonder is there any tool to monitor or check website with time based and scrape new content into my database then I will convert to embeddings and add into my vector database?
I mostly use website content crawler on apify store for adding the content into pinecone which is my vector database for passing these vector into langchain LLM to create chatbot using Python. I wonder is there any tool to monitor or check website with time based and scrape new content into my database then I will convert to embeddings and add into my vector database?
