How to save network requests made by the webpage I am scraping?
Hello, the scraping that I'm trying to do is not of actual content on the page, but rather network requests. I don't need anything too fancy - I just basically want to dump everything one would see in the Network tab of their browser's Inspect tool. I tried searching through the docs, but"request" gives back a lot of unrelated stuff since that word is pretty central to how Crawlee works :).
If there is another tool that would be more appropriate for this, please let me know. I still need to be able to deal with JS-heavy pages, and I still need to be able to follow links. It's just that the end product I need is requests, not page elements.
If there is another tool that would be more appropriate for this, please let me know. I still need to be able to deal with JS-heavy pages, and I still need to be able to follow links. It's just that the end product I need is requests, not page elements.
