Logs management - ELK
Hello, is it possible somehow to make Crawlee to output for example INFO level logs to the console and DEBUG level logs to a file ?
I want to implement the ELK stack with crawlee and looking for the best solution.
I'm open to any ideas.
Context : I have a few hundreds of crawlers written with crawlee on AWS Batch, and I want to have a dashboard where I get various stats from logs. Outputting all the logs (DEBUG) to CloudWatch would increase the cost by a lot.
I want to implement the ELK stack with crawlee and looking for the best solution.
I'm open to any ideas.
Context : I have a few hundreds of crawlers written with crawlee on AWS Batch, and I want to have a dashboard where I get various stats from logs. Outputting all the logs (DEBUG) to CloudWatch would increase the cost by a lot.
