brilliant-limeB

dedupe queue

Hey All, When running a Task from the console, is there any way to dedupe the queue? I imagine that would be kind of a default, and an easy toggle in the config screen. I am havnig the same URL crawled twice per run.
Was this page helpful?