stuck-chocolate•2y ago

Code refactoring - reusing a common handler in multiple crawlers while keeping code hints

I have some code that looks something like this:

router.addHandler('log-in', async ({...}) => {...})

router.addHandler('log-in', async ({...}) => {...})

I want to reuse this handler in multiple crawlers. I could of course simply create the function log_in in some other file, but then I would use code hints for page/request etc. is there a better way to go about reusing the handler?

9 Replies

flat-fuchsia•2y ago

Why tf has this not been answered 🥲 The documentation even says

We would most likely go even further and split even the routes into separate files.

We would most likely go even further and split even the routes into separate files.

and yet this is very unclear without losing linting and useful dev features. I'm upset. https://crawlee.dev/docs/introduction/refactoring#splitting-your-code-into-multiple-files

stuck-chocolateOP•2y ago

Yea... I ended up adding and removing a dummy router while developing to deal with this. Hacky solution, but it works. When I asked this question, I was super new to the lovely thing called javascript. From what I've seen in the past two months - this isn't as much an issue with Crawlee, as it is an issue with Javascript in general... just a badly designed language. I haven't tried this myself, but I think moving to typescript might solve this issue. (I can't use it since I'm limited by my team ;p)

flat-fuchsia•2y ago

I am using typescript and have XP across both. I think your idea is a bit of a red herring 😄

Pepa J•2y ago

router.addHandler takes label and handler function as a parameter. You may import both from different file.

stuck-chocolateOP•2y ago

welp, was just a silly idea XD hoped you could use typescript to define a "handler function" type that you could reference in the new file. but again, I know nothing about typescript beyond it being a typed version of javascript ;p I'm just importing ideas from python/c++ is handler function a type? that you can import? thats the issue. that a generic function won't allow for dev tools to work properly.

flat-fuchsia•2y ago

you lose linting doing this as the types are unknown. causes hacky looking code in editor, I didn't say its not possible... just not great dev friendliness

MEE6•2y ago

@TKR👑 just advanced to level 1! Thanks for your contributions! 🎉

flat-fuchsia•2y ago

you'd be manually redefining the function signature? so any change to the handler you'd have to account for in your cloned type. dodging the problem imo 🥲

stuck-chocolateOP•2y ago

yupppp, I agree, its far from a perfect solution. but idk at this point.

Code refactoring - reusing a common handler in multiple crawlers while keeping code hints

Did you find this page helpful?