Apify & Crawlee

AC

Apify & Crawlee

This is the official developer community of Apify and Crawlee.

Join

crawlee-js

apify-platform

crawlee-python

💻hire-freelancers

🚀actor-promotion

💫feature-request

💻creators-and-apify

🗣general-chat

🎁giveaways

programming-memes

🌐apify-announcements

🕷crawlee-announcements

👥community

Why is no code showing in the web IDE?

Anyone know why my code is not showing in the web IDE? I've adjusted the Dockerfile, so that might be why. What should I change? ```Dockerfile Specify the base Docker image. You can read more about the available images at https://crawlee.dev/docs/guides/docker-images...
Solution:
@Louis Deconinck I checked your case and unfortunately this is a feature of apify-cli, to upload large directories as a zip archive and therefore it doesn't show up in the WebIDE. It is mentioned in the doc....
No description

About Actor Quality

On the Actor Quality page, I received a message about creating an output_schema.json. My Actor’s output schema is already embedded inside actor.json. The documentation clearly states that it can be included either inside actor.json or in a separate output_schema.json. https://docs.apify.com/platform/actors/development/actor-definition/output-schema So, to improve Actor quality, does it need to have a separate output_schema.json file?...
No description

Output schema

This is my output schema: ```json { "actorOutputSchemaVersion": 1, "title": "Output schema of Skool Group Scraper",...
No description

Actor build keep running clean build although I choose build with cache

Can anyone explain how building actor work? Although I only choose build option which decalre that it will build with Docker cache layer, it's keep building from scratch, cannot find out why this happended

Error: Actor with identifier "my-actor" is already on the platform and was modified there since ...

I get this error when trying to run apify push on my actor. What is this? The cli suggests to use --force to override but while that makes the deploy succeed, my actor still fails in production with the error Error: Cannot find module '/usr/src/app/dist/main.js'. Help....

Output schema for actor quality score is a mistake?

The actor quality score takes the presence of an output schema into consideration (one of the suggested improvements). I think this is a mistake. Most (of my) actors use the default dataset store, technically not requiring an output schema. I assume that the intention is probably to check for the presence of a dataset schema and not output schema. Is that correct? This is at least what the description seems to imply. "Help users quickly see what data your Actor produces. Without an output schema, users see raw data that's hard to interpret. Add a schema to display results in organized tables with clear field names and types. All Actors should have one defined, even if they don't produce a dataset. Link: https://docs.apify.com/platform/actors/development/actor-definition/output-schema?fpr=7p4wu "...

$1M Challenge Apify Apps - Note Sure Which One?

Hi Guys. Question about the $1M challenge. In the https://apify.com/challenge it says under FAQ, "Are any types of Actors not eligible for rewards?" -> and it specifieis "Rental or pay-per-result (PPR) Actors. More on pricing models here." BUT in the backend "insights" -> "actor quality" is says eligble only for "The Actor has either a Pay per result or Pay per event pricing model". Can you explain which is the correct one? ANd in the terms (https://docs.apify.com/legal/challenge-terms-and-condi...
Solution:
Sorry that was a mistake. Only PPE actors are eligible for the challenge....

Configure pricing from /1000 to /1

I have about 10 actors that need this changed because they are AI based and are expected to give 1-5 output rows per run, users find that pricing intimidating, How can I configure this so it displays as $0.5/1 and not $500/1000 This is making it harder for users to pay for the actor since they get scared of the price...
Solution:
The one-time events are already implemented, but I suppose you will have more than one result, so it won't really work for you. The feature you request should be released next week. The Apify team is already testing it, but they have to do it thoroughly since it touches prices......

Synthetic start event for PPE actor: First 5 seconds of compute cost covered by Apify?

When reading the developer documentation it is mentioned that Apify will cover the compute unit cost of the first 5 seconds of every Actor run when you use the apify-actor-start event. Link: https://docs.apify.com/platform/actors/publishing/monetize/pay-per-event?fpr=7p4wu#use-synthetic-start-event-apify-actor-start Is it then correct to assume that Apify won't cover the compute unit cost of the first 5 seconds of every Actor run if we don't use the apify-actor-start event in our PPE actor? I'm asking this, because there was some confusion about this in the last Creator Cantina call. @Saurav Jain recommended not to charge a start event to be more competitive as we get the first 5 seconds for free. But as far as I understand it, we only get the first 5 seconds for free if we charge the apify-actor-start event....

Need help changing my actor price

Greetings please I need help modifying my actor price, I tried changing it, but I’m being restricted since it can only be changed once per month. Unfortunately i need to adjust the price sooner, I would greatly appreciate it if I could get help. Thanks...

payout

Hi Apify support team, I've noticed that my PayPal payout address is wrong. I thought that I would have until October 21 to update my billing details. But the invoice was automatically approved today (October 14)....
No description

Bug while paginating in debugging section

While going to next page in debugging section with 1 actor it works fine but as we take more than 2 actor and paginate it just gets deselected.

Question about renewal price for my plan

Hi, I’m currently on the Creator plan, which includes the $500 bonus credit. I’d like someone from the team to review my current usage and estimate what my monthly cost will look like once this plan and the bonus credit end. Thanks!...

Output Schema

I am trying to add output schema for my actor as specified in https://docs.apify.com/platform/actors/development/actor-definition/output-schema I created a new output_schema.json file and linked it in my actor.json ```...
Solution:
Output schema and dataset schema are two separate things. You want to implement this: https://docs.apify.com/platform/actors/development/actor-definition/dataset-schema?fpr=7p4wu

Unexpected losses issue

I am starting to have big losses but the insights are inaccurate and i can't be sure which actor fails. On debugging section there is not a run that justifies that.
Solution:
I found the cause of the issue (thanks to support). My actor remained idle for over an hour for no apparent reason. Although decreasing the timeout period can mitigate losses, losses still occur. I have now implemented a rule in my code for the actor to exit if it remains idle for more than five minutes.

issue with current pricing model and storage

I’m building an actor that scrapes media (images, audio, and short videos) and stores them in key–value storage. According to the monetization guide, storage costs are deducted from my revenue. This makes it hard to set a price, because storage usage is very dynamic: files can be very small or very large, and storage is billed by the hour while users might keep their files forever and never delete them. I feel like I’m missing something here. Does anyone have an idea how to handle this?...
Solution:
I believe that the only storage costs that affecting the developer are the costs related to the data stored in the developer storage, and the costs related to the data an actor stores in the users storage only during the actor run for PPR and PPE. For PPR and PPE the users pay for keeping the data in the storage after the run is finished, the price is not too much though.

Doubt Regarding Pinterest Actor and “More Ideas” Tab Scraping

I had a question regarding the Pinterest actor — does it also scrape the “More Ideas” posts from Pinterest boards? I’m referring to the suggested pins that appear under the “More Ideas” tab, not the actual pins saved on the board....
No description

Google Maps Scraper: extendOutputFunction failing to skip existing placeIds

Hello everyone, I'm using the Google Maps Scraper Actor in a workflow and attempting to implement logic to skip places that I have already collected in previous runs, based on their unique placeId. My setup involves querying a database for all previously collected placeIds and passing that list to the Actor's input as an array called customData.ignoreIds. The Problem: The filtering logic is correctly defined in the Actor's input, but the Actor is still collecting and outputting places whose placeId is present in the ignoreIds list. I have verified the workflow is sequential (Database $\rightarrow$ JS $\rightarrow$ Actor).I have verified the ignoreIds array correctly contains the duplicate IDs in the Actor's input.The Actor log shows no errors related to the custom function.This suggests the extendOutputFunction is either not being executed or is failing silently within the Actor's environment. My Implementation "async ({ item, data, customData }) => {" +...

Pay as you go billing

I am on the free plan running a task that doesn't need many resources, however more than the $5 a month that are free. As I understood I would be paying as I go for any additional usage. However my task has now just stopped after the $5 was reached. How do I set it to pay as I go? The $35 plan would be way too much for me, as I need $10 max
Solution:
You would have to upgrade to the starter plan here: https://apify.com/pricing?fpr=7p4wu For each actor you can also limit the maximum spending. You do this under the input tab....
Next