feat(logging): Log buffering support for Logj42 and Logback

[phipag]

Summary

Docs preview: https://dealn7fl31ram.cloudfront.net/core/logging/#buffering-logs

This PR introduces a log buffering feature to Powertools for AWS Lambda (Java). Log buffering is the mechanism of buffering lower level logs and only flush them (to STDOUT on Lambda) in case of error or manually if needed. It is a good way to reduce the noise caused by e.g. DEBUG logs and can be seen as a more elaborate way of "log sampling".

While the other Powertools runtimes (Python, TypeScript, .NET) follow a programmatic approach to configuring log buffering, this PR uses the concept of "Appenders" in the corresponding logging backend (log4j2, logback). This means that log buffering is configured using the configuration files of the respective logging framework which is compatible with the way how structured JSON logging is already implemented. This also means that log buffering can be used independent of the @Logging annotation (except for automatic flushing on exceptions) and it is also compatible with any other Appender. The idea is to wrap an arbitrary existing Appender (e.g. a STDOUT JSON structured appender) with the BufferingAppender which will enable log buffering with zero code changes. Here is an example for log4j2:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
    <Appenders>
        <Console name="JsonAppender" target="SYSTEM_OUT">
            <JsonTemplateLayout eventTemplateUri="classpath:LambdaJsonLayout.json" />
        </Console>
        <!-- New BufferingAppender wrapping existing JsonAppender emitting Lambda structured JSON logs -->
        <BufferingAppender name="BufferedAppender" bufferAtVerbosity="DEBUG" maxBytes="20480">
            <AppenderRef ref="JsonAppender" />
        </BufferingAppender>
    </Appenders>
    <Loggers>
        <Root level="debug">
            <!-- Reference the BufferedAppender wrapping the JsonAppender instead of the JsonAppender directly -->
            <AppenderRef ref="BufferedAppender" />
        </Root>
    </Loggers>
</Configuration>

Note: This XML based configuration supersedes the original design proposal of @BufferingConfig made in the feature request issue (see below).

Changes

• Implement log buffering feature
• Extend E2E tests to test for both log4j2 and logback implementations (we only had log4j2 before)
• Update examples to use log buffering feature
• Add section in documentation about log buffering
• Make buffering compatible with GraalVM

Issue number: #2095

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud