molecular-blue•2y ago
Anyone knows if there is a difference
Anyone knows if there is a difference between puppeteer’s evaluate() and chromium’s devtool console? I am currently scraping a site and doing a fetch api manually via the devtool console works perfectly, but im being blocked when using puppeteer’s evaluate(), despite using the same fetch function. Wonder if anyone could help me with this?
3 Replies
Hi, which page? Are you being blocked when using evaluate or before that the page does not even load?
rare-sapphire•2y ago
Shopee.co.id
Being blocked when using evaluate
But when i spin up a non headless puppeteer and manually call fetch api using the devtool, it gives valid response, 200
Using evaluate im being blocked 403
Some sites have much stricter blocking rules for their API vs their pages. So can be that they just tagged you as automated browser.