ordinary-sapphireO
Apify & Crawlee3y ago
12 replies
ordinary-sapphire

web scrapper create 1 file instead of multiple output for pagination

I used this article https://docs.apify.com/academy/advanced-web-scraping/scraping-paginated-sites#define-and-enqueue-pivot-ranges to scrap data of multiple pages, when I run apify run I get 20 different json files. How can I combine all the data in 1 json file for all the pages?
Learn how to extract all of a website's listings even if they limit the number of results pages. See code examples for setting up your scraper.
Scraping paginated sites | Apify Documentation
Was this page helpful?