Apify Discord Mirror

Updated 3 months ago

enqueue urls / request queue not being unique

At a glance

The community member is experiencing an issue where they are seeing the same URLs being run twice. A community member responds that this is not the default behavior, and suggests that the equality of two requests is determined by their uniqueKey properties. By changing this property value, the community member could potentially process the same URLs multiple times, as long as their uniqueKey values are unique. The community member asks if the original poster is explicitly setting this property.

I'm seeing a lot of the same exact URL's being ran twice? Any ideas?
M
1 comment
Hi. Hm...that is not the default behavior πŸ€”.
The equality of two requests is determined via comparison of their uniqueKey properties. By changing this property value, you could be able to process the same URLs multiple times (if their uniqueKey values were unique ofc). Do you explicitly set this property?

See this.
Add a reply
Sign up and join the conversation on Discord