fierDeToiMonGrandF
Apify & Crawlee•4w ago•
5 replies
fierDeToiMonGrand

Is there a way to enqueue a request without actually making an HTTP request?

What I’m trying to do is leverage Crawlee’s request queue + uniqueKey de-duplication, but I don’t want to hit a real URL.

Conceptually, I just want to “switch routes” (change the label) and trigger another operation/state in the crawler, while still using the queue’s uniqueness guarantees.

Is this possible, or is a real URL always required when adding a request?


Currently getting
 Reclaiming failed request back to the list or queue. Expected `RequestOptions.url` to be of type `string` but received type `undefined`


I want something like that
import { v4 as uuidv4 } from 'uuid';

const itemRequests: Partial<Source>[] = dataResults.map((result) => {
    const itemKey = result.item.key; // generic key
    return {
        uniqueKey: request.userData.saveOnlyUniqueItems ? itemKey : uuidv4(),
        label: labels.SPECIFIC_ITEM, // generic label
        userData: {
            ...request.userData,
            itemData: result.item, // pass the data generically
        },
    };
});
Was this page helpful?