Can i get 403 status
Hi.
I guess this question might be a bit dumb, but i wanted to ask how does the crawlee work with requests?
If i want to access some particular website using pure request or axios i get 403 error, but with cralwee cheerioCrawler, i get the result i want.
I figured out the retry mechanism and the session rotation has to do something with it since it happened few times with my use case.
I know it's too much to ask, but just wondering like if its going trough some proxy's, how are user agents handled, tls handshake etc.
I'm asking all of this since I'm wondering if the website could block me again somehow?
Sorry for a newbie question again
I guess this question might be a bit dumb, but i wanted to ask how does the crawlee work with requests?
If i want to access some particular website using pure request or axios i get 403 error, but with cralwee cheerioCrawler, i get the result i want.
I figured out the retry mechanism and the session rotation has to do something with it since it happened few times with my use case.
I know it's too much to ask, but just wondering like if its going trough some proxy's, how are user agents handled, tls handshake etc.
I'm asking all of this since I'm wondering if the website could block me again somehow?
Sorry for a newbie question again
