Mmilitary-pink
Request works in Postman but doesnt work with Cheerio Crawler, request object headers empty
Dear all, I am trying to scrap data from a public ip. For some reason cheeriocrawler is not getting the data back but in postman I could easily get the data. Proxy ip is whitelisted because I am using the same ip for postman and for cheerio.
Postman does add some default headers but when I look at my request object the headers are empty. Does someone knows at which points cheerio sets the headers and generate some fingerprints and how can I see them ?
any help would be highly appreciated. Thanks
Postman does add some default headers but when I look at my request object the headers are empty. Does someone knows at which points cheerio sets the headers and generate some fingerprints and how can I see them ?
Request {
id: 'OBTRQI5zvA4aIJ9',
url: 'https://someapi.com',
loadedUrl: 'https://someapi.com',
uniqueKey: '22586062-3f0d-40be-b499-f1a00261b5d3',
method: 'GET',
payload: undefined,
noRetry: false,
retryCount: 0,
errorMessages: [],
headers: {},
userData: [Getter/Setter],
handledAt: undefined
}Request {
id: 'OBTRQI5zvA4aIJ9',
url: 'https://someapi.com',
loadedUrl: 'https://someapi.com',
uniqueKey: '22586062-3f0d-40be-b499-f1a00261b5d3',
method: 'GET',
payload: undefined,
noRetry: false,
retryCount: 0,
errorMessages: [],
headers: {},
userData: [Getter/Setter],
handledAt: undefined
}any help would be highly appreciated. Thanks
