6 replies

Configure Apify Proxy urls in a Crawlee Playwright crawler

I've been trying to use Apify proxies in my Crawlee crawler, but have had no luck with it, always getting an

net::ERR_TUNNEL_CONNECTION_FAILED

net::ERR_TUNNEL_CONNECTION_FAILED

error. Evidently I'm doing something wrong, but the documentation has been extremely unhelpful, and it's been very hard to find tutorials on the matter. The relevant sections of code are as follows:

import {
  PlaywrightCrawler,
  ProxyConfiguration,
} from 'crawlee';

const proxyUrls = [
  `http://groups-BUYPROXIES94952:${this.apifyToken}@proxy.apify.com:8000`,
  `http://groups-RESIDENTIAL:${this.apifyToken}@proxy.apify.com:8000`,
];

const crawler = new PlaywrightCrawler({
  requestHandler: router,
  maxConcurrency: 10,
  navigationTimeoutSecs: 180,
  requestHandlerTimeoutSecs: 180,
  proxyConfiguration: new ProxyConfiguration({
    proxyUrls,
  }),
});

import {
  PlaywrightCrawler,
  ProxyConfiguration,
} from 'crawlee';

const proxyUrls = [
  `http://groups-BUYPROXIES94952:${this.apifyToken}@proxy.apify.com:8000`,
  `http://groups-RESIDENTIAL:${this.apifyToken}@proxy.apify.com:8000`,
];

const crawler = new PlaywrightCrawler({
  requestHandler: router,
  maxConcurrency: 10,
  navigationTimeoutSecs: 180,
  requestHandlerTimeoutSecs: 180,
  proxyConfiguration: new ProxyConfiguration({
    proxyUrls,
  }),
});

I really need some help with this. Thanks in advance.

Configure Apify Proxy urls in a Crawlee Playwright crawler

Similar Threads