adverse-sapphire•2y ago

Custom LoggerText implementation not handling objects

hey folks, I had to write up a custom logger implementation because I needed to store the logs in a file and rotate them as well. I have extended LoggerText and have implemented the methods as well and while its working its skipping a lot of debug messages that we get with the inbuilt logger as well as not being able to handle objects being passed to it. I have looked at the source code for it and have stuck to it as much as I could. I would really appreciate it if someone could point out whats wrong with it. code: https://pastebin.com/tpGxyM0P actual output: https://pastebin.com/ABtC8QSa

Pastebin

import { LoggerText, LogLevel } from 'crawlee';import * as winston ...

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

Pastebin

{"level":"debug","message":"No 'persistStateKeyValueStoreId' option...

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

5 Replies

adverse-sapphireOP•2y ago

on further debugging, it looks like in the log method, all of the data being sent is treated as exception ok I fixed the debug stuff by flipping the order of function variables and making it like its there in the _log func but the json is adding \ everywhere which I know is to skip double quotes but its not like that in the crawlee implementation, is there something I can do to fix it? also I am calling _parseException just like in _log but its not parsing the exception at all and instead has undefined and I am passing an exception along like this

try{
../code
}catch(error:any){
//any fixes for this any? it looks really ugly but ts wont let me pass it in without any
log.error('error',error)
}

try{
../code
}catch(error:any){
//any fixes for this any? it looks really ugly but ts wont let me pass it in without any
log.error('error',error)
}

adverse-sapphireOP•2y ago

Leaving this for others but this is how I fixed it https://pastebin.com/3awiV7ce

Pastebin

import { LoggerText, LogLevel } from 'crawlee';import * as winston ...

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

adverse-sapphireOP•2y ago

only issue I can find with crawlee's implementation is, I had to break out winston's logger in a separate function because crawlee's log would keep overriding my routes.ts settings if I imported a different file which was using a crawlee log too, Not sure if its supposed to be a bug or anything as for double encoding, that was winston's json formatter shenanigans, we are already encoding our json into strings inside _log and winston was further encoding it as per my setup

Lukas Krivka•2y ago

Thanks, I will pass it to the team to make example

adverse-sapphireOP•2y ago

yeah, I fixed it here: https://discord.com/channels/801163717915574323/1182635626579906560/1183228038339637398 , just leaving it for others

Custom LoggerText implementation not handling objects

Did you find this page helpful?