sunny-green•3y ago
how to clear request queue without stoping crawler
I want crawler to run constantly and i want to remove old links to clear memory and storage
1 Reply
sunny-greenOP•3y ago
what i mean to say is that when task is fulfilled, it is saved in cache until you drop the queue; what i want to do is that i don't want it to cache fulfilled task