Configure Apify Proxy urls in a Crawlee Playwright crawler

I've been trying to use Apify proxies in my Crawlee crawler, but have had no luck with it, always getting an net::ERR_TUNNEL_CONNECTION_FAILEDnet::ERR_TUNNEL_CONNECTION_FAILED error. Evidently I'm doing something wrong, but the documentation has been extremely unhelpful, and it's been very hard to find tutorials on the matter. The relevant sections of code are as follows:

import {
  PlaywrightCrawler,
  ProxyConfiguration,
} from 'crawlee';

const proxyUrls = [
  `http://groups-BUYPROXIES94952:${this.apifyToken}@proxy.apify.com:8000`,
  `http://groups-RESIDENTIAL:${this.apifyToken}@proxy.apify.com:8000`,
];

const crawler = new PlaywrightCrawler({
  requestHandler: router,
  maxConcurrency: 10,
  navigationTimeoutSecs: 180,
  requestHandlerTimeoutSecs: 180,
  proxyConfiguration: new ProxyConfiguration({
    proxyUrls,
  }),
});

import {
  PlaywrightCrawler,
  ProxyConfiguration,
} from 'crawlee';

const proxyUrls = [
  `http://groups-BUYPROXIES94952:${this.apifyToken}@proxy.apify.com:8000`,
  `http://groups-RESIDENTIAL:${this.apifyToken}@proxy.apify.com:8000`,
];

const crawler = new PlaywrightCrawler({
  requestHandler: router,
  maxConcurrency: 10,
  navigationTimeoutSecs: 180,
  requestHandlerTimeoutSecs: 180,
  proxyConfiguration: new ProxyConfiguration({
    proxyUrls,
  }),
});

I really need some help with this. Thanks in advance.

Apify & Crawlee•4y ago•

6 replies

full-green

Configure Apify Proxy urls in a Crawlee Playwright crawler

import {
  PlaywrightCrawler,
  ProxyConfiguration,
} from 'crawlee';

const proxyUrls = [
  `http://groups-BUYPROXIES94952:${this.apifyToken}@proxy.apify.com:8000`,
  `http://groups-RESIDENTIAL:${this.apifyToken}@proxy.apify.com:8000`,
];

const crawler = new PlaywrightCrawler({
  requestHandler: router,
  maxConcurrency: 10,
  navigationTimeoutSecs: 180,
  requestHandlerTimeoutSecs: 180,
  proxyConfiguration: new ProxyConfiguration({
    proxyUrls,
  }),
});

import {
  PlaywrightCrawler,
  ProxyConfiguration,
} from 'crawlee';

const proxyUrls = [
  `http://groups-BUYPROXIES94952:${this.apifyToken}@proxy.apify.com:8000`,
  `http://groups-RESIDENTIAL:${this.apifyToken}@proxy.apify.com:8000`,
];

const crawler = new PlaywrightCrawler({
  requestHandler: router,
  maxConcurrency: 10,
  navigationTimeoutSecs: 180,
  requestHandlerTimeoutSecs: 180,
  proxyConfiguration: new ProxyConfiguration({
    proxyUrls,
  }),
});

I really need some help with this. Thanks in advance.

Continue the conversation

Join the Discord to ask follow-up questions and connect with the community

A&C

Apify & Crawlee

This is the official developer community of Apify and Crawlee.

15,108 Members

Join

Configure Apify Proxy urls in a Crawlee Playwright crawler

Configure Apify Proxy urls in a Crawlee Playwright crawler

Continue the conversation

Apify & Crawlee

Continue the conversation

Apify & Crawlee

Similar Threads