How to override `maxRequestRetries` error log

there is a function

    protected async _handleFailedRequestHandler(crawlingContext: Context, error: Error): Promise<void> {
        // Always log the last error regardless if the user provided a failedRequestHandler
        const { id, url, method, uniqueKey } = crawlingContext.request;
        const message = this._getMessageFromError(error, true);

        this.log.error(`Request failed and reached maximum retries. ${message}`, { id, url, method, uniqueKey });

        if (this.failedRequestHandler) {
            await this._tagUserHandlerError(() =>
                this.failedRequestHandler?.(this._augmentContextWithDeprecatedError(crawlingContext, error), error),
            );
        }
    }

    protected async _handleFailedRequestHandler(crawlingContext: Context, error: Error): Promise<void> {
        // Always log the last error regardless if the user provided a failedRequestHandler
        const { id, url, method, uniqueKey } = crawlingContext.request;
        const message = this._getMessageFromError(error, true);

        this.log.error(`Request failed and reached maximum retries. ${message}`, { id, url, method, uniqueKey });

        if (this.failedRequestHandler) {
            await this._tagUserHandlerError(() =>
                this.failedRequestHandler?.(this._augmentContextWithDeprecatedError(crawlingContext, error), error),
            );
        }
    }

that is triggered once maxRequestRetries is done, how could I override this log message with my own? I don't want to see whole stacktrace in my logs, I just want to notify that there is an error and details can be found under some id in DB

should I disable logs for error and handle them manually?

Apify & Crawlee•2y ago•

3 replies

rubber-blue

How to override `maxRequestRetries` error log

there is a function

    protected async _handleFailedRequestHandler(crawlingContext: Context, error: Error): Promise<void> {
        // Always log the last error regardless if the user provided a failedRequestHandler
        const { id, url, method, uniqueKey } = crawlingContext.request;
        const message = this._getMessageFromError(error, true);

        this.log.error(`Request failed and reached maximum retries. ${message}`, { id, url, method, uniqueKey });

        if (this.failedRequestHandler) {
            await this._tagUserHandlerError(() =>
                this.failedRequestHandler?.(this._augmentContextWithDeprecatedError(crawlingContext, error), error),
            );
        }
    }

    protected async _handleFailedRequestHandler(crawlingContext: Context, error: Error): Promise<void> {
        // Always log the last error regardless if the user provided a failedRequestHandler
        const { id, url, method, uniqueKey } = crawlingContext.request;
        const message = this._getMessageFromError(error, true);

        this.log.error(`Request failed and reached maximum retries. ${message}`, { id, url, method, uniqueKey });

        if (this.failedRequestHandler) {
            await this._tagUserHandlerError(() =>
                this.failedRequestHandler?.(this._augmentContextWithDeprecatedError(crawlingContext, error), error),
            );
        }
    }

Continue the conversation

Join the Discord to ask follow-up questions and connect with the community

A&C

Apify & Crawlee

This is the official developer community of Apify and Crawlee.

14,735 Members

Join

How to override `maxRequestRetries` error log

How to override `maxRequestRetries` error log

Continue the conversation

Apify & Crawlee

Continue the conversation

Apify & Crawlee

Similar Threads