verbal-limeV
Apify & Crawlee•3y ago•
6 replies
verbal-lime

RequestRouter not using the `log` instance passed in its respective Playwright crawler

Hey folks, I have a bit of a weird use case here, I need to scrape multiple sites and I know the recommended approach is to use a single scraper and pass in the urls to the starting queue

But issue with that is, the logging gets messed up with all the different site logs getting mixed together, to fix that I created multiple loggers for it but the issue I am facing is,

when we import 
log
 from 
crawlee
 it looks like its shared between all the places that has imported 
log
 too, and I have a custom logging implementation which is different for each site (mainly in the format of logs), and I need crawlee to respect that.

So to fix this I looked at the docs and 
createPlaywrightCrawler
 has an option for log where you can pass in your own logger to it, and while it uses the correct format for crawlee's internal logging, it does nothing to the logs which I explicitly log in my routehandler.

so How do I either separate log instances for each instance, or make my route handler respect the logger i have passed when I created the crawler
Was this page helpful?