thenetaji•2mo ago

Rotate country in proxy for each request

can we rotate country for proxy without relaunching crawlee? I need to use specific country for every url, without relaunching crawlee everytime.

2 Replies

luigi.ruocco•2mo ago

Yeah, you can do something like this

const crawler = new CheerioCrawler(
    {
        proxyConfiguration: await Actor.createProxyConfiguration({ groups: ['RESIDENTIAL'], countryCode: 'US' }),
        requestHandler: async ({ proxyInfo }) => { log.info(proxyInfo?.url ?? ''); },
        preNavigationHooks: [
            async (context) => {
                context.proxyInfo = await (await Actor.createProxyConfiguration({
                    groups: ['RESIDENTIAL'],
                    countryCode: 'IT',
                }))?.newProxyInfo();
            },
        ],
    },
);

const crawler = new CheerioCrawler(
    {
        proxyConfiguration: await Actor.createProxyConfiguration({ groups: ['RESIDENTIAL'], countryCode: 'US' }),
        requestHandler: async ({ proxyInfo }) => { log.info(proxyInfo?.url ?? ''); },
        preNavigationHooks: [
            async (context) => {
                context.proxyInfo = await (await Actor.createProxyConfiguration({
                    groups: ['RESIDENTIAL'],
                    countryCode: 'IT',
                }))?.newProxyInfo();
            },
        ],
    },
);

thenetajiOP•2mo ago

Thanks man. I would check it out

Rotate country in proxy for each request

Did you find this page helpful?