Apify & Crawlee

This is the official developer community of Apify and Crawlee.

Join

Apify & Crawlee

This is the official developer community of Apify and Crawlee.

Join

crawlee-js

apify-platform

crawlee-python

💻hire-freelancers

🚀actor-promotion

💫feature-request

💻creators-and-apify

🗣general-chat

🎁giveaways

programming-memes

🌐apify-announcements

🕷crawlee-announcements

👥community

correct-apricot

6/24/2024

Unable to setup apify actor using python-scrapy template in Windows

Hi Team, I'm new to Apify. I'm trying to setup Apify actor using python-scrapy template. But I'm getting following error. ``` C:\Users\Guest User\Desktop>apify create my-actor -t python-scrapy Info: Making sure that Apify CLI is up to date......

national-gold

6/19/2024

Question regarding Subscription

Hello I'd like to inquire about your LinkedIn profile scraper API (https://console.apify.com/actors/PEgClm7RgRD7YO94b/). The pricing page mentions a $25/month + usage fees, but I'm curious if that's a recurring charge regardless of usage, or if I'll only be billed $25 once I've exhausted the previous month's allotment. Thanks...

quickest-silver

6/14/2024

how do i do a simple http url with onclick jv buttons

trying to do a headless java onclick an pull current info from http website, any idea why it's not working.... ive tried scrapy selenium etc...

foreign-sapphire

6/10/2024

Instagram scraper // Error no public data

Hi everyone 🙂 I'm currently developing a Python-based solution using your API. After a few successful tests throughout the day, I have the impression that “nothing works anymore”. Indeed, concerning the “instagram-scraper” API, I'm getting the following message telling me that no more data is contained in my “results” object.

environmental-rose

6/6/2024

Billing

They sent me a bill even I cancel subscription already. Please help....

solid-orange

6/4/2024

Using the Actor instance across multiple files

Hi, how can i use the same actor instance across multiple functions defined in seperate files? for example, if I define an Actor in the main.py file as: async with Actor:...

eastern-cyan

6/4/2024

Using Chrome Extensions in apify selenium

Hey, I've been trying to make a LinkedIn scraper, and all of the basic code is done, however, one thing I've used in other hubs such as repl.it was an extension to find emails, as well as one to solve captchas. However I can't appear to be able to use them in apify, because the actor has a limit of 3 mb for storage. Is there any way I can install the extension in a selenium webdriver in apify?

other-emerald

5/24/2024

common actor code

Hello, do you have any suggestions on how to share code between actors without using public repositories like pypi or github, instead only using apify platform/web IDE? Some crazy idea: import from key value store apify internal pypi...

ambitious-aqua

5/13/2024

Proxy Not Working

Hello. I recently tried to use an Apify proxy in my Python Playwright script, and it's giving me this error: ``` playwright._impl._api_types.Error: net::ERR_TUNNEL_CONNECTION_FAILED at WEBSITE =========================== logs ===========================...

extended-salmon

5/6/2024

Multiple Datasets with different schemes

Hi how can i handle different dataset output schemes there is no docs for doing that ....

quickest-silver

4/17/2024

Download images and store in apify

Hi, I read the documentation on keyvalue pair storage, but did not quite really fiure out how to use it to store images, videos or audio using python sdk. Is there any example code that I can reference to?

harsh-harlequin

4/11/2024

Error handling / best practices

Hello, I am using pre-built actors in my application. I use them like this to create the dataset: ```python client = ApifyClientAsync(token=settings.APIFY_API_TOKEN) run = await client.actor(actor.value).start(run_input=run_input)...

deep-jade

4/10/2024

Issue with the payment

This is the error I am facing everytime, while making payment for a subscription plan, how to resolve it?

ambitious-aqua

4/7/2024

Apify with langchain and openAI

Hi, I'm using Langchains community apify packageto load directly datasets for better answers. But the results are miserably... So I try find a way to set and modify the model that is generating answers with temperature, context role and so on. I was studying the docs and code form the Apify Wrapper but there is no option to set and modify the model. So there must be the chance to set it higher in the tree. Any ideas?

deep-jade

4/6/2024

ApifyAPIError: Input is not valid: Field input.usernames is required

I am trying to fetch profile data from a csv list and i have defined a row as the input for the second actor. The row contains already scrapped usernames of some Instagram accounts. I have initialized the client as Apifyclient(API) and and defined some fields that i want to extract from the data i scrape from this actor. The problem lies in run_input, as i have a list called usernames which contains usernames for the accounts/profiles i want to scrape. Upon implementing a loop to go through the list, i am receiving an error which says the following: ...

wise-white

4/5/2024

I am getting this error for all my python actor

apify_client._errors.ApifyApiError: Unexpected error: <html><body><h1>503 Service Unavailable</h1> No server is available to handle this request....

deep-jade

4/1/2024

Scrapper working well locally and not on apify

someone had this problem

deep-jade

4/1/2024

add_request with same url but different payload

do you have any example of adding a request to the requestQueue with using use_extended_unique_key because I am using the same url for every request the only change is the payload ? I have this error TypeError: Unicode-objects must be encoded before hashing with this code for i in range(1,max_index + 1) : # Update the page number in params_query #json_data['indices']['products']['paging']['index'] = int(next_index) json_data = json.dumps(json_data)...

rare-sapphire

3/28/2024

Fb accounts

Is there any way to scrape FB accounts.

!!!Joefree!!! 👑

3/23/2024

Python API multiple unnamed create_dataset

On APIFY SDK there is no method for creating-dataset multiple dataset or (there is ?) reference: https://docs.apify.com/sdk/python/reference/class/Dataset#open API: https://docs.apify.com/api/v2#/reference/datasets/dataset-collection/create-dataset ...

Previous Next