Scetchy scrolling distance (apify/web-scraper)
Hi,
I'm using the apify/web-scraper to scrape a website. The pages don't initially load all the content, you need to scroll down to see more.
Now i have set the scroll distance setting to the highest number possible but still it only works half of the time.
Is there any way to improve the scroll behavior? Right now works on only let's say 50% of the requests, on what pages it works is different each run. Is there another way to control this functionality?
Thanks,
Bob
I'm using the apify/web-scraper to scrape a website. The pages don't initially load all the content, you need to scroll down to see more.
Now i have set the scroll distance setting to the highest number possible but still it only works half of the time.
Is there any way to improve the scroll behavior? Right now works on only let's say 50% of the requests, on what pages it works is different each run. Is there another way to control this functionality?
Thanks,
Bob
