Apify

Apify Discord Mirror

Updated 5 months ago

How to disable crawlee log?

At a glance

The community member encountered an issue while using the Example Usage from the Crawlee/Playwright library, where a request to https://httpbin.org/status/404 resulted in an error. The community member is interested in disabling the Crawlee/Playwright log that displays this error. In the comments, another community member provided a solution that involves using log.setLevel(log.LEVELS.OFF) to disable the log, but noted that it is not a very pretty solution.

Useful resources
LLeMoussel
·
If I do Example Usage (https://crawlee.dev/api/playwright-crawler#example-usage) with this Url: https://httpbin.org/status/404, I got this output :
2022-10-11 08:52:58.931 WARN PlaywrightCrawler: Reclaiming failed request back to the list or queue. page.goto: net::ERR_HTTP_RESPONSE_CODE_FAILURE at https://httpbin.org/status/404 =========================== logs =========================== navigating to "https://httpbin.org/status/404", waiting until "load" ============================================================ {"id":"sOcDKee4CooEnLF","url":"https://httpbin.org/status/404","retryCount":1} 2022-10-11 08:53:03.429 ERROR PlaywrightCrawler: Request failed and reached maximum retries. page.goto: net::ERR_HTTP_RESPONSE_CODE_FAILURE at https://httpbin.org/status/404 =========================== logs =========================== navigating to "https://httpbin.org/status/404", waiting until "load" ============================================================ at gotoExtended (c:\Users\HERNOUX-06523\Desktop\Dev\NodeJS\test-crawlee\node_modules\@crawlee\playwright\internals\utils\playwright-utils.js:149:17) at PlaywrightCrawler._navigationHandler (c:\Users\HERNOUX-06523\Desktop\Dev\NodeJS\test-crawlee\node_modules\@crawlee\playwright\internals\playwright-crawler.js:105:52) at PlaywrightCrawler._handleNavigation (c:\Users\HERNOUX-06523\Desktop\Dev\NodeJS\test-crawlee\node_modules\@crawlee\browser\internals\browser-crawler.js:268:51) at async PlaywrightCrawler._runRequestHandler (c:\Users\HERNOUX-06523\Desktop\Dev\NodeJS\test-crawlee\node_modules\@crawlee\browser\internals\browser-crawler.js:215:17) at async PlaywrightCrawler._runRequestHandler (c:\Users\HERNOUX-06523\Desktop\Dev\NodeJS\test-crawlee\node_modules\@crawlee\playwright\internals\playwright-crawler.js:102:9) ....
Is it possible to disable this Crawlee/Playwright log?
L
t
2 comments
LLeMoussel
a solution that I don't find very pretty:
Plain Text
import {
   log
} from 'crawlee';

const crawlLog = log.child({ prefix: 'TestCrawlee' })
crawlLog.setOptions({
    logger: new LoggerText({ skipTime: false }),
});
log.setLevel(log.LEVELS.OFF);
tthek1tten
From what I'm aware, that's unfortunately the way to do it
Add a reply
Sign up and join the conversation on Discord