Duplicate results from multiple runs

At a glance

The community member is running public actors, such as Google Maps and Yelp extractors, and is concerned about avoiding duplicate results when changing settings and running the actors again for different locations. The community member asks if there is a way to avoid re-running URLs that have already been processed in previous runs.

In the comments, another community member explains that there is no generic solution for this, and it would need to be implemented in the actor itself. They mention an actor called "abort-and-resurrect" that allows aborting, transforming input, and resurrecting input of an active run, which might solve the issue, but they caution that it could lead to unexpected behavior depending on the actor.

Useful resources

BBOBPG

Hey,

When I run an actor, I can resurrect it, and I assume there will be no duplicate results.

But when I want to change some settings, it's a new run, it probably re-runs urls I already ran in the previous run.

Is there a way to avoid this? Let's say an actor for businesses (google, yelp, yell etc) runs for London, and i start a new run for a new location or a location within london that might have overlap.
How do I avoid paying again for duplicates? Is there a setting to avoid re-running urls that have already run in a previous runs?

Thanks,

3 comments

vvojtechmaslan

Hello , are you using public actors or your own ones?

BBOBPG

Public ones, e.g. the google maps extractor, the yelp extractor etc

vvojtechmaslan

Unfortunately, there is no generic solution for this. This would have to be implemented in the Actor itself.

There is this actor: https://apify.com/mnmkng/abort-and-resurrect that allows you to abort, trasnform input and resurrect input of an active run, would this solve your issue? As this changes input on fly, this might lead to unexpected behaviour. Again, depends on th Actor.

Add a reply

Apify Discord Mirror

Duplicate results from multiple runs