Apify & Crawlee

This is the official developer community of Apify and Crawlee.

Join

Apify & Crawlee

This is the official developer community of Apify and Crawlee.

Join

crawlee-js

apify-platform

crawlee-python

💻hire-freelancers

🚀actor-promotion

💫feature-request

💻creators-and-apify

🗣general-chat

🎁giveaways

programming-memes

🌐apify-announcements

🕷crawlee-announcements

👥community

azzouzana

7/17/2025

Fix broken links in the Apify sdk versions documentation page

Documentations' links are all broken (notice the undefined keyword when you hover on any of them - bottom left of attached screenshot). Also release notes links point to 404

333mmm333

7/17/2025

Help using apify proxy without sdk

I'm trying to use apify proxys without sdk, using my own framework. But i keep getting Proxy Authentication Required errors. ```golang func NewApifyProxyClientFromEnv() (*ApifyProxyClient, error) { hostname := os.Getenv("APIFY_PROXY_HOSTNAME") port := os.Getenv("APIFY_PROXY_PORT")...

Solution:

never mind, i've just realized i was getting a 407 proxy error and i've fixed it and now it is 403, the apify datacenter proxy does not work with the site

Mujeeb

7/15/2025

Actor Development

Hey there, I just reverse-engineered the API of a big e-commerce giant because scraping was totally impossible from their website due to hard blockades by captchas and a very sophisticated bot system by their developer team(Hats off to them). I had a look at Apify and found out it is not available here, so I am here to find out how I can become an actor developer. What is the procedure? How can I change my already written Python code to act as an actor? and what about the commission, etc? Thanks and nice to meet you to guys Mujeeb...

zac

7/15/2025

Avoiding Crawler Detection

Hi folks, I have a crawlee playwright script that I can run locally without being blocked. However, when I deploy and run it on apify I'm getting blocked. Do you have any suggestions on how to avoid it? Here's the error I'm seeing in the apify console: WARN PlaywrightCrawler: Reclaiming failed request back to the list or queue. Navigation timed out after 60 seconds....

azzouzana

7/12/2025

fix this alert link

Fix this "Learn more" link, it's missing a "/" (it's displayed in this page https://console.apify.com/actors/development)...

Mark Kalnitski

7/12/2025

Scraper doesn't trigger the integration

Hey guys, the Instagram Scraper by Apify doesn't send a payload to my inbound webhook despite it pretends to attempt. Here is a screen recording. I appreciate your support. https://vento.so/view/d49d4210-5762-4f84-8432-2380267ad41c?utm_medium=share Best regards...

fierDeToiMonGrand

7/11/2025

My actor is Generating negative revenue

Hello, I have an actor that is generating negative revenue and I would like to know which runs are causing that and fix it. However the analytics tool that Apfiy provide is really limited what are my options?...

Xeno

7/10/2025

Small bug - My Actors page

Clicking Learn how takes you to https://docs.apify.complatform/actors/publishing/monetize#pay-per-event-ppe-pricing-model instead of https://docs.apify.com/platform/actors/publishing/monetize#pay-per-event-ppe-pricing-model Typo, it's missing a / between .com and platform...

emdzej

7/10/2025

Layout has a bug.

Hello there! There is a CSS bug on the layout page at https://docs.apify.com/sdk/js (macOS 15, Firefox 140). Hopefully, this is the right place to report it!

RapidTech1898

7/6/2025

Not working on all elements in the request_queue?

Hello - i try to run the following apify-code and as you can see in my example code below i would like to create a queue with 6 elements / combinations: [('electrician', 'Southampton,England,United Kingdom'), ('electrician', 'Luton,England,United Kingdom'), ('electrician', 'Portsmouth,England,United Kingdom'), ('accountant', 'Southampton,England,United Kingdom'), ('accountant', 'Luton,England,United Kingdom'), ('accountant', 'Portsmouth,England,United Kingdom')] But when i run the code only the first element is executed from the queue and then the actor is exiting:...

RapidTech1898

7/5/2025

See all the requests in a queue?

Hello - i build a request-queue using the following code: for i,e in enumerate(inpCombinations):
newReq = Request.from_url(f"{baseLink}#{i}") newReq.user_data = {"search": list(e), "excluded": inpExcludedWords}...

7/5/2025

What should my "Personal ID" be when setting up billing for my Actor.

Just wondering what piece of ID should be entered for the Business ID / Personal ID section for an Canada, Ontario resident wanting to rent out my Actor.

merc

7/4/2025

google maps scrapper faster response?

Hi i've just started using Apify to scrape some data for my web app. Essentially I'm trying to replace usage of serpAPI for scraping google maps places and videos as the cost is abit too much. so was trying out google maps scrapper by Compass, which works for me, but i'm curious if theres a way to get quicker response maybe by having the actor on standby mode (although this can be costly as well but considering) so that we wont have to wait for the cold start of docker on every request? The app im building requires an on demand data, based on a user's query, eg. "hiking in Japan", then show about 10 places min (with 1 image per result). currently it takes more than 15s using the scraper. so was wondering if there's any way to make the run time faster apart from the actor standby mode or optimistic loading of those places? the one by Huncho is $50/m so I'm trying to see if can make it work with google maps scraper instead...

Samer

7/3/2025

Understand how to use actors

I'm planning to use a Facebook post scraper. As I can see, it costs $5.00/month + usage. But when I want to rent the Actor, I can see that I have to have a paid plan with at least 39$ a month. Every month $5 will be automatically deducted from your Apify prepaid usage until canceled. I will use this actor very little, so I'm wondering if I can buy a paid plan for one month, then if I cancel it. Will I have the 39$ as a balance that I can use 5$ each month? Or will the 39$ reset at the next month? Thanks...

ken00865

7/1/2025

camoufox in headed mode

I'm having a issue with camoufox in headed mode and the actor abruptly closing without any error and without finishing the job. One thing to notice is that I've started my development with the python-empty template....

RapidTech1898

6/29/2025

Getting Pydantic warning when building Requeset Queue

I try to build a request queue using the following code: (the user-data i want to add is a tuple) request_queue = await Actor.open_request_queue()
for e in inpCombinations: ...

RapidTech1898

6/29/2025

Change headless mode?

Hello - i am currently running the actor locally in VSCode in python - How and where can i change if the Actor is running headless or not with this "Actor.config.headless" parameter Where can i change this to False for example?...

azzouzana

6/27/2025

Could someone please help me get my referral account approved

Basically the title

Christopher

6/26/2025

Golf Data..scraping app

i'm working on a project for an individual who works with golf course data. all things related to the course, yardages, tee times, superintendents, slope ratings, etc. older gentlemen has been manually "scraping" data for years. i'm new to the scraping tools, but it sounds like Apify would be a great solution to run and gather all this data automatically. ...

Mariela

6/25/2025

Hello! I am interested in a marketing collaboration.

Hello! I am interested in a marketing collaboration. My suggestion is https://blog.apify.com/what-is-an-anonymous-proxy/ Anchor: brand protection software...

Previous Next