correct-apricot•2y ago

Crawlee Router as a folder with different files for each Handler

Hey, everyone! 👋 Is it possible to create a folder named routes in my project and then construct export const router in index.ts file by adding all my handlers from different files? Like this

// Many imports...

export const router = createPlaywrightRouter();

router.addDefaultHandler(defaultHandler);
router.addHandler('catalog-page', catalogHandler);
router.addHandler('review-page', reviewsHandler);

// Many imports...

export const router = createPlaywrightRouter();

router.addDefaultHandler(defaultHandler);
router.addHandler('catalog-page', catalogHandler);
router.addHandler('review-page', reviewsHandler);

And if this is possible, what should be the type of the handler in a separate file?

4 Replies

correct-apricotOP•2y ago

From the sources, I've understood that handler should have type

handler: (ctx: Omit<Context, 'request'> & { request: Request<UserData> }) => Awaitable<void>

handler: (ctx: Omit<Context, 'request'> & { request: Request<UserData> }) => Awaitable<void>

but is it possible to use this type or any alias in my own files? Because it seems that Context is defined as a type parameter of the Router class

Lukas Krivka•2y ago

This seems to work

type Handler = (ctx: Omit<PlaywrightCrawlingContext<Dictionary>, 'request'> & { request: Request<Dictionary>; }) => Awaitable<void>
const handleRoute: Handler = async ({ page }) => {
    console.log(await page.title());
};

const router = createPlaywrightRouter();
router.addHandler('handler', handleRoute);

type Handler = (ctx: Omit<PlaywrightCrawlingContext<Dictionary>, 'request'> & { request: Request<Dictionary>; }) => Awaitable<void>
const handleRoute: Handler = async ({ page }) => {
    console.log(await page.title());
};

const router = createPlaywrightRouter();
router.addHandler('handler', handleRoute);

correct-apricotOP•2y ago

Thanks a lot Lukas! That's indeed what I was looking for

MEE6•2y ago

@Mike just advanced to level 1! Thanks for your contributions! 🎉

Crawlee Router as a folder with different files for each Handler

Did you find this page helpful?