Apify Discord Mirror

Updated 5 months ago

socks5 passwore protected proxies

At a glance

The community member is asking how to use socks5 proxies with the Crawlee library, and how to handle password-protected proxies. The comments suggest trying different approaches with Playwright and Puppeteer, such as using the proxy option or passing proxy arguments to the launch options. However, there is no explicitly marked answer, and one community member raises a question about how this might affect Crawlee's proxy rotation algorithm.

Useful resources

ccurioussoul

Dear all, how can I use socks5 proxies with crawlee ? Also in general of the proxy is password protected how to put it in proxyUrl ?
I didnt find any exmaple to use password protected proxies and socks5 proxies are not supported by defualt.

Anyway to get around it ?

Best Regards

5 comments

MMojo

Hi, we had an interesting conversation here, it's still not working for me on firefox/chromium with playwright thought https://discord.com/channels/801163717915574323/999250964554981446/1080132532495986819

MMojo

I tried this with playwright/firefox:

Plain Text

 launchContext: {
        launchOptions: {
            proxy: {
                server: `socks5=https://${PROXY_DOMAIN}:${PROXY_PORT}`,
                username: 'username',
                password: 'password'
            } 
        },
    },

and this one with playwright/chromium and puppeteer:

Plain Text

 
launchContext: {
        launchOptions: {
            args: [ `--proxy-server=socks5=https://${PROXY_USERNAME}:${PROXY_PASSWORD}@${PROXY_DOMAIN}:${PROXY_PORT}` ]
        }
    },

MMojo

The second option using args in puppeteer launch works perfectly with my own implementation of my chrome scraper

ccurioussoul

Thanks for the help Mojo. But wouldnt that affect the algorithm by which crawlee rotate the proxies ?

Here the proxy is bound to the browser but what if I want to rotate it ?

MMojo

good question

Add a reply