got many 429 status code when crawled the target site,even though proxies. How to optimise my code?
Hi,guys. I am a python coder but not good at nodejs. I make a crawler to bulk check the infomation by the crawlee.
This is my option :
useSessionPool: true,
useIncognitoPages: true,
and I using the Residential Proxies .
But I found that the ip of some pages as same as the others.
I want launch diffrent browser context in each target url . But I don't know how to do that.
Anybody could help me ?
Sorry for my poor English .
Thanks
This is my option :
useSessionPool: true,
useIncognitoPages: true,
and I using the Residential Proxies .
But I found that the ip of some pages as same as the others.
I want launch diffrent browser context in each target url . But I don't know how to do that.
Anybody could help me ?
Sorry for my poor English .
Thanks
