More meaningful error than ERR_TUNNEL_CONNECTION_FAILED
Hi There.
I am using a proxy to crawl some sites and encounter a ERR_TUNNEL_CONNECTION_FAILED error.
I am using brightData as my proxy service. If i was to curl my proxy endpoint I get a meaningful errror. For example
HTTP/1.1 502 Could not resolve host demo-site.reckoniq.coms
x-brd-err-code: target_40001
x-brd-err-msg: Could not resolve host demo-site.reckoniq.coms
X-Luminati-Error: Could not resolve host demo-site.reckoniq.coms
x-brd-error: Could not resolve host demo-site.reckoniq.coms
Proxy-Connection: close
or
HTTP/1.1 403 Forbidden
x-brd-err-code: policy_20050
x-brd-err-msg: Forbidden: target site requires special permission. You are trying to access a target site which is not permitted by our compliance policy. In order to gain access you may need to undergo a KYC process, you can do so by filling in the form: https://brightdata.com/cp/kyc If you have already completed the KYC approval, please contact your account manager for further details.
X-Luminati-Error: Forbidden: target site requires special permission. Contact BrightData for assistance
x-brd-error: Forbidden: target site requires special permission. Contact BrightData for assistance
Proxy-Connection: close
I am wondering how i can surface those errors in crawlee rather than just getting
ERROR PlaywrightCrawler: Request failed and reached maximum retries. Error: Detected a session error, rotating session...
page.goto: net::ERR_TUNNEL_CONNECTION_FAILED at https://demo-site.reckoniq.coms/
There is no response in the error handler so things are not makinging it that far
Thanks for any help
I am using a proxy to crawl some sites and encounter a ERR_TUNNEL_CONNECTION_FAILED error.
I am using brightData as my proxy service. If i was to curl my proxy endpoint I get a meaningful errror. For example
HTTP/1.1 502 Could not resolve host demo-site.reckoniq.coms
x-brd-err-code: target_40001
x-brd-err-msg: Could not resolve host demo-site.reckoniq.coms
X-Luminati-Error: Could not resolve host demo-site.reckoniq.coms
x-brd-error: Could not resolve host demo-site.reckoniq.coms
Proxy-Connection: close
or
HTTP/1.1 403 Forbidden
x-brd-err-code: policy_20050
x-brd-err-msg: Forbidden: target site requires special permission. You are trying to access a target site which is not permitted by our compliance policy. In order to gain access you may need to undergo a KYC process, you can do so by filling in the form: https://brightdata.com/cp/kyc If you have already completed the KYC approval, please contact your account manager for further details.
X-Luminati-Error: Forbidden: target site requires special permission. Contact BrightData for assistance
x-brd-error: Forbidden: target site requires special permission. Contact BrightData for assistance
Proxy-Connection: close
I am wondering how i can surface those errors in crawlee rather than just getting
ERROR PlaywrightCrawler: Request failed and reached maximum retries. Error: Detected a session error, rotating session...
page.goto: net::ERR_TUNNEL_CONNECTION_FAILED at https://demo-site.reckoniq.coms/
There is no response in the error handler so things are not makinging it that far
Thanks for any help
World's largest proxy service with a residential proxy network of 72M IPs worldwide and proxy management interface for zero coding. Start a 7-day free trial »
