TypeError: init() got an unexpected keyword argument 'follow_redirects'

I am using Apify Client to scrape information in a cron job written in airflow dag. When i try to execute the task file manually, it works fine but when i do so from the airflow UI, it throws this error. I tried to upgrade the version of

httpx

httpx

to 0.24 but apify client support 0.23

  APPIFY_CLIENT = ApifyClient(config('APIFY_CLIENT_TOKEN'))
File "/usr/share/pyshared/venv/airflow/lib/python3.8/site-packages/apify_client/client.py", line 126, in __init__
    self.http_client = _HTTPClient(
File "/usr/share/pyshared/venv/airflow/lib/python3.8/site-packages/apify_client/_http_client.py", line 59, in __init__
    self.httpx_client = httpx.Client(headers=headers, follow_redirects=True, timeout=timeout_secs)

TypeError: __init__() got an unexpected keyword argument 'follow_redirects'

  APPIFY_CLIENT = ApifyClient(config('APIFY_CLIENT_TOKEN'))
File "/usr/share/pyshared/venv/airflow/lib/python3.8/site-packages/apify_client/client.py", line 126, in __init__
    self.http_client = _HTTPClient(
File "/usr/share/pyshared/venv/airflow/lib/python3.8/site-packages/apify_client/_http_client.py", line 59, in __init__
    self.httpx_client = httpx.Client(headers=headers, follow_redirects=True, timeout=timeout_secs)

TypeError: __init__() got an unexpected keyword argument 'follow_redirects'

Alexey Udovydchenko•5/3/23, 2:13 AM

its airflow issue, client not getting valid input and you need somehow to exclude

!!!Joefree!!! 👑•5/3/23, 4:31 AM

Have you tried change to allow_redirects ?

dry-scarletOP•5/3/23, 5:16 AM

@!!!Joefree!!!

@Alexey Udovydchenko i removed the follow_redirects from the source code and installed on my system via pip. It works fine after that