Scraping Bet365's live odds

Hey everyone, I am trying to scrape data from games that are in-play on Bet365. Unfortunately, their site blocks all requests via Cloudflare, and connecting to the websockets without getting data from the site first is impossible. I would also prefer not to use Selenium as it is hard to scale and the data is prone to errors. They are also are able to detect Selenium and seem to block most typical workarounds, so it doesn't really even seem possible lol. Any help would be greatly appreciated.

Alexey Udovydchenko•12/1/23, 3:24 AM

Playwright + Firefox should be enough to bypass CF protection, you can try it by https://apify.com/apify/playwright-scraper and if approach works imho just run actor from your python code

primary-violet•12/1/23, 6:59 AM

The solution is to directly access your computer's memory or intercept packets on the network with a sniffer! We can say that these ways are undetectable, Forget playwright, puppeteer or selenium, it will eventually break...

primary-violet•12/1/23, 7:11 AM

sniffer_video.zip6.29MB

primary-violet•12/1/23, 7:12 AM

Python (scapy lib) + Bluestacks + Wireshark