Improve Logging Serialization #6891

RobertKeyser · 2025-10-30T18:58:15Z

Description Of Changes

This PR addresses some logging issues where certain logs were not getting logged in the loguru format, but rather whatever native logging format their libraries were set up with. Most notably this should cover Celery.

Doing so helps to improve logging consistency, which is especially helpful when ingesting logs into a logging service like Datadog.

Code Changes

Short-circuit when celery tries to configure its own logger
InterceptHandler to snag logs and log them with loguru
Force color to be disabled when logging with json
Rework a few print() statements to use loguru instead.

Steps to Confirm

Here are some examples of what the worker logs looked like and what they'll look like now.

BEFORE


2025-10-30 14:08:55 {"text": "2025-10-30 19:08:54.999 | INFO     | fides.api.worker:log_celery_setup:102 - Celery connection setup complete | {'celery_details': {'hostname': 'celery@5c3d786ecf84', 'version': '5.5.3 (immunity)', 'app': 'fides.api.tasks:0xffff6d1defe0', 'transport': 'redis://:@redis:6379//', 'results': 'redis://:@redis:6379/', 'concurrency': '2', 'queues': 'fides|fidesops.messaging|fides.privacy_preferences|fides.dsr|fidesplus.consent_webhooks|fidesplus.discovery_monitors_detection|fidesplus.discovery_monitors_classification|fidesplus.discovery_monitors_promotion'}}\n", "record": {"elapsed": {"repr": "0:00:16.177022", "seconds": 16.177022}, "exception": null, "extra": {"celery_details": {"hostname": "celery@5c3d786ecf84", "version": "5.5.3 (immunity)", "app": "fides.api.tasks:0xffff6d1defe0", "transport": "redis://:@redis:6379//", "results": "redis://:@redis:6379/", "concurrency": "2", "queues": "fides|fidesops.messaging|fides.privacy_preferences|fides.dsr|fidesplus.consent_webhooks|fidesplus.discovery_monitors_detection|fidesplus.discovery_monitors_classification|fidesplus.discovery_monitors_promotion"}}, "file": {"name": "init.py", "path": "/fidesplus/ethyca-fides/src/fides/api/worker/init.py"}, "function": "log_celery_setup", "level": {"icon": "ℹ️", "name": "INFO", "no": 20}, "line": 102, "message": "Celery connection setup complete", "module": "init", "name": "fides.api.worker", "process": {"id": 70, "name": "MainProcess"}, "thread": {"id": 281473004183584, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:08:54.999411+00:00", "timestamp": 1761851334.999411}}}

2025-10-30 14:08:56 [2025-10-30 19:08:56,147: INFO/MainProcess] Connected to redis://:**@redis:6379//

2025-10-30 14:08:56 [2025-10-30 19:08:56,153: INFO/MainProcess] mingle: searching for neighbors

2025-10-30 14:08:57 [2025-10-30 19:08:57,162: INFO/MainProcess] mingle: all alone

2025-10-30 14:08:57 [2025-10-30 19:08:57,175: INFO/MainProcess] celery@5c3d786ecf84 ready.

AFTER


2025-10-30 14:13:55 {"text": "2025-10-30 19:13:55.228 | DEBUG 2025-10-30 14:13:55 {"text": "2025-10-30 19:13:55.257 | INFO 2025-10-30 14:13:55 {"text": "2025-10-30 19:13:55.260 | DEBUG 2025-10-30 14:13:55 {"text": "2025-10-30 19:13:55.262 | DEBUG 2025-10-30 14:13:55 {"text": "2025-10-30 19:13:55.292 | DEBUG 2025-10-30 14:13:55 {"text": "2025-10-30 19:13:55.293 | DEBUG 2025-10-30 14:13:55 {"text": "2025-10-30 19:13:55.293 | INFO 2025-10-30 14:13:56 {"text": "2025-10-30 19:13:56.326 | INFO 2025-10-30 14:13:56 {"text": "2025-10-30 19:13:56.327 | DEBUG 2025-10-30 14:13:56 {"text": "2025-10-30 19:13:56.328 | DEBUG 2025-10-30 14:13:56 {"text": "2025-10-30 19:13:56.341 | DEBUG 2025-10-30 14:13:56 {"text": "2025-10-30 19:13:56.342 | DEBUG 2025-10-30 14:13:56 {"text": "2025-10-30 19:13:56.347 | DEBUG 2025-10-30 14:13:56 {"text": "2025-10-30 19:13:56.348 | DEBUG 2025-10-30 14:13:56 {"text": "2025-10-30 19:13:56.387 | DEBUG 2025-10-30 14:13:56 {"text": "2025-10-30 19:13:56.387 | DEBUG 2025-10-30 14:13:56 {"text": "2025-10-30 19:13:56.392 | DEBUG 2025-10-30 14:13:56 {"text": "2025-10-30 19:13:56.392 | DEBUG 2025-10-30 14:13:56 {"text": "2025-10-30 19:13:56.392 | DEBUG 2025-10-30 14:13:56 {"text": "2025-10-30 19:13:56.397 | INFO

| celery.bootsteps:_debug:259 - | Consumer: Starting Connection\n", "record": {"elapsed": {"repr": "0:00:14.838384", "seconds": 14.838384}, "exception": null, "extra": {}, "file": {"name": "bootsteps.py", "path": "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py"}, "function": "_debug", "level": {"icon": "🐞", "name": "DEBUG", "no": 10}, "line": 259, "message": "| Consumer: Starting Connection", "module": "bootsteps", "name": "celery.bootsteps", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:55.228718+00:00", "timestamp": 1761851635.228718}}}
 | celery.worker.consumer.connection:start:22 - Connected to redis://:@redis:6379//\n", "record": {"elapsed": {"repr": "0:00:14.867344", "seconds": 14.867344}, "exception": null, "extra": {}, "file": {"name": "connection.py", "path": "/usr/local/lib/python3.10/site-packages/celery/worker/consumer/connection.py"}, "function": "start", "level": {"icon": "ℹ️", "name": "INFO", "no": 20}, "line": 22, "message": "Connected to redis://:@redis:6379//", "module": "connection", "name": "celery.worker.consumer.connection", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:55.257678+00:00", "timestamp": 1761851635.257678}}}
 | celery.bootsteps:start:117 - ^-- substep ok\n", "record": {"elapsed": {"repr": "0:00:14.870206", "seconds": 14.870206}, "exception": null, "extra": {}, "file": {"name": "bootsteps.py", "path": "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py"}, "function": "start", "level": {"icon": "🐞", "name": "DEBUG", "no": 10}, "line": 117, "message": "^-- substep ok", "module": "bootsteps", "name": "celery.bootsteps", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:55.260540+00:00", "timestamp": 1761851635.26054}}}
 | celery.bootsteps:_debug:259 - | Consumer: Starting Events\n", "record": {"elapsed": {"repr": "0:00:14.871811", "seconds": 14.871811}, "exception": null, "extra": {}, "file": {"name": "bootsteps.py", "path": "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py"}, "function": "_debug", "level": {"icon": "🐞", "name": "DEBUG", "no": 10}, "line": 259, "message": "| Consumer: Starting Events", "module": "bootsteps", "name": "celery.bootsteps", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:55.262145+00:00", "timestamp": 1761851635.262145}}}
 | celery.bootsteps:start:117 - ^-- substep ok\n", "record": {"elapsed": {"repr": "0:00:14.901767", "seconds": 14.901767}, "exception": null, "extra": {}, "file": {"name": "bootsteps.py", "path": "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py"}, "function": "start", "level": {"icon": "🐞", "name": "DEBUG", "no": 10}, "line": 117, "message": "^-- substep ok", "module": "bootsteps", "name": "celery.bootsteps", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:55.292101+00:00", "timestamp": 1761851635.292101}}}
 | celery.bootsteps:_debug:259 - | Consumer: Starting Mingle\n", "record": {"elapsed": {"repr": "0:00:14.902812", "seconds": 14.902812}, "exception": null, "extra": {}, "file": {"name": "bootsteps.py", "path": "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py"}, "function": "_debug", "level": {"icon": "🐞", "name": "DEBUG", "no": 10}, "line": 259, "message": "| Consumer: Starting Mingle", "module": "bootsteps", "name": "celery.bootsteps", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:55.293146+00:00", "timestamp": 1761851635.293146}}}
 | celery.worker.consumer.mingle:sync:40 - mingle: searching for neighbors\n", "record": {"elapsed": {"repr": "0:00:14.903514", "seconds": 14.903514}, "exception": null, "extra": {}, "file": {"name": "mingle.py", "path": "/usr/local/lib/python3.10/site-packages/celery/worker/consumer/mingle.py"}, "function": "sync", "level": {"icon": "ℹ️", "name": "INFO", "no": 20}, "line": 40, "message": "mingle: searching for neighbors", "module": "mingle", "name": "celery.worker.consumer.mingle", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:55.293848+00:00", "timestamp": 1761851635.293848}}}
 | celery.worker.consumer.mingle:sync:49 - mingle: all alone\n", "record": {"elapsed": {"repr": "0:00:15.935805", "seconds": 15.935805}, "exception": null, "extra": {}, "file": {"name": "mingle.py", "path": "/usr/local/lib/python3.10/site-packages/celery/worker/consumer/mingle.py"}, "function": "sync", "level": {"icon": "ℹ️", "name": "INFO", "no": 20}, "line": 49, "message": "mingle: all alone", "module": "mingle", "name": "celery.worker.consumer.mingle", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:56.326139+00:00", "timestamp": 1761851636.326139}}}
 | celery.bootsteps:start:117 - ^-- substep ok\n", "record": {"elapsed": {"repr": "0:00:15.937072", "seconds": 15.937072}, "exception": null, "extra": {}, "file": {"name": "bootsteps.py", "path": "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py"}, "function": "start", "level": {"icon": "🐞", "name": "DEBUG", "no": 10}, "line": 117, "message": "^-- substep ok", "module": "bootsteps", "name": "celery.bootsteps", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:56.327406+00:00", "timestamp": 1761851636.327406}}}
 | celery.bootsteps:_debug:259 - | Consumer: Starting Gossip\n", "record": {"elapsed": {"repr": "0:00:15.938101", "seconds": 15.938101}, "exception": null, "extra": {}, "file": {"name": "bootsteps.py", "path": "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py"}, "function": "_debug", "level": {"icon": "🐞", "name": "DEBUG", "no": 10}, "line": 259, "message": "| Consumer: Starting Gossip", "module": "bootsteps", "name": "celery.bootsteps", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:56.328435+00:00", "timestamp": 1761851636.328435}}}
 | celery.bootsteps:start:117 - ^-- substep ok\n", "record": {"elapsed": {"repr": "0:00:15.951336", "seconds": 15.951336}, "exception": null, "extra": {}, "file": {"name": "bootsteps.py", "path": "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py"}, "function": "start", "level": {"icon": "🐞", "name": "DEBUG", "no": 10}, "line": 117, "message": "^-- substep ok", "module": "bootsteps", "name": "celery.bootsteps", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:56.341670+00:00", "timestamp": 1761851636.34167}}}
 | celery.bootsteps:_debug:259 - | Consumer: Starting Heart\n", "record": {"elapsed": {"repr": "0:00:15.951901", "seconds": 15.951901}, "exception": null, "extra": {}, "file": {"name": "bootsteps.py", "path": "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py"}, "function": "_debug", "level": {"icon": "🐞", "name": "DEBUG", "no": 10}, "line": 259, "message": "| Consumer: Starting Heart", "module": "bootsteps", "name": "celery.bootsteps", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:56.342235+00:00", "timestamp": 1761851636.342235}}}
 | celery.bootsteps:start:117 - ^-- substep ok\n", "record": {"elapsed": {"repr": "0:00:15.957161", "seconds": 15.957161}, "exception": null, "extra": {}, "file": {"name": "bootsteps.py", "path": "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py"}, "function": "start", "level": {"icon": "🐞", "name": "DEBUG", "no": 10}, "line": 117, "message": "^-- substep ok", "module": "bootsteps", "name": "celery.bootsteps", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:56.347495+00:00", "timestamp": 1761851636.347495}}}
 | celery.bootsteps:_debug:259 - | Consumer: Starting Tasks\n", "record": {"elapsed": {"repr": "0:00:15.958020", "seconds": 15.95802}, "exception": null, "extra": {}, "file": {"name": "bootsteps.py", "path": "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py"}, "function": "_debug", "level": {"icon": "🐞", "name": "DEBUG", "no": 10}, "line": 259, "message": "| Consumer: Starting Tasks", "module": "bootsteps", "name": "celery.bootsteps", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:56.348354+00:00", "timestamp": 1761851636.348354}}}
 | celery.bootsteps:start:117 - ^-- substep ok\n", "record": {"elapsed": {"repr": "0:00:15.996838", "seconds": 15.996838}, "exception": null, "extra": {}, "file": {"name": "bootsteps.py", "path": "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py"}, "function": "start", "level": {"icon": "🐞", "name": "DEBUG", "no": 10}, "line": 117, "message": "^-- substep ok", "module": "bootsteps", "name": "celery.bootsteps", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:56.387172+00:00", "timestamp": 1761851636.387172}}}
 | celery.bootsteps:_debug:259 - | Consumer: Starting Control\n", "record": {"elapsed": {"repr": "0:00:15.997428", "seconds": 15.997428}, "exception": null, "extra": {}, "file": {"name": "bootsteps.py", "path": "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py"}, "function": "_debug", "level": {"icon": "🐞", "name": "DEBUG", "no": 10}, "line": 259, "message": "| Consumer: Starting Control", "module": "bootsteps", "name": "celery.bootsteps", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:56.387762+00:00", "timestamp": 1761851636.387762}}}
 | celery.bootsteps:start:117 - ^-- substep ok\n", "record": {"elapsed": {"repr": "0:00:16.002032", "seconds": 16.002032}, "exception": null, "extra": {}, "file": {"name": "bootsteps.py", "path": "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py"}, "function": "start", "level": {"icon": "🐞", "name": "DEBUG", "no": 10}, "line": 117, "message": "^-- substep ok", "module": "bootsteps", "name": "celery.bootsteps", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:56.392366+00:00", "timestamp": 1761851636.392366}}}
 | celery.bootsteps:_debug:259 - | Consumer: Starting event loop\n", "record": {"elapsed": {"repr": "0:00:16.002270", "seconds": 16.00227}, "exception": null, "extra": {}, "file": {"name": "bootsteps.py", "path": "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py"}, "function": "_debug", "level": {"icon": "🐞", "name": "DEBUG", "no": 10}, "line": 259, "message": "| Consumer: Starting event loop", "module": "bootsteps", "name": "celery.bootsteps", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:56.392604+00:00", "timestamp": 1761851636.392604}}}
 | celery.bootsteps:_debug:259 - | Worker: Hub.register Pool...\n", "record": {"elapsed": {"repr": "0:00:16.002615", "seconds": 16.002615}, "exception": null, "extra": {}, "file": {"name": "bootsteps.py", "path": "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py"}, "function": "_debug", "level": {"icon": "🐞", "name": "DEBUG", "no": 10}, "line": 259, "message": "| Worker: Hub.register Pool...", "module": "bootsteps", "name": "celery.bootsteps", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:56.392949+00:00", "timestamp": 1761851636.392949}}}
 | celery.apps.worker:on_consumer_ready:176 - celery@2427a933df37 ready.\n", "record": {"elapsed": {"repr": "0:00:16.007365", "seconds": 16.007365}, "exception": null, "extra": {}, "file": {"name": "worker.py", "path": "/usr/local/lib/python3.10/site-packages/celery/apps/worker.py"}, "function": "on_consumer_ready", "level": {"icon": "ℹ️", "name": "INFO", "no": 20}, "line": 176, "message": "celery@2427a933df37 ready.", "module": "worker", "name": "celery.apps.worker", "process": {"id": 69, "name": "MainProcess"}, "thread": {"id": 281473580527648, "name": "MainThread"}, "time": {"repr": "2025-10-30 19:13:56.397699+00:00", "timestamp": 1761851636.397699}}}

Pre-Merge Checklist

vercel · 2025-10-30T18:58:21Z

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments

Project	Deployment	Preview	Comments	Updated (UTC)
fides-plus-nightly	Ignored	Preview		Nov 7, 2025 4:20pm
fides-privacy-center	Ignored			Nov 7, 2025 4:20pm

RobertKeyser · 2025-10-30T19:27:38Z

src/fides/api/util/logger.py

    extra_dict = {
        **standard_dict,
        "format": log_format + " | <dim>{extra}</dim>",
-        "filter": lambda logRecord: bool(logRecord["extra"]),


Pretty sure this was always truthy. Before I swapped it for the has_custom_extra function, the Celery mingle messages weren't showing up.

greptile-apps

Greptile Overview

Greptile Summary

This PR improves logging consistency by intercepting standard library logs from Celery, SQLAlchemy, and other libraries, routing them through Loguru for unified formatting.

Key changes:

Added InterceptHandler class to capture stdlib logging and route through Loguru
Connected to Celery's setup_logging signal to prevent it from overriding Loguru configuration
Fixed color output when using JSON serialization (colors are now disabled for JSON)
Replaced print() statements with proper log.info() calls in config and database modules
Added comprehensive test coverage for the InterceptHandler

Potential issue:

The frame traversal logic in InterceptHandler.emit() may not correctly identify the caller when logs originate from logging module subfiles (like logging/handlers.py), as it only checks against logging.__file__ which points to logging/__init__.py

Confidence Score: 4/5

This PR is safe to merge with minimal risk after addressing the frame traversal logic issue
The PR successfully improves logging consistency with good test coverage and clean refactoring. The only concern is the frame traversal logic in InterceptHandler which may not correctly identify caller info in all scenarios when logs come from logging module subfiles, though this is unlikely to cause runtime failures - just potentially incorrect file/line info in logs
src/fides/api/util/logger.py - Review the frame traversal logic in InterceptHandler.emit()

Important Files Changed

File Analysis

Filename	Score	Overview
src/fides/api/util/logger.py	4/5	Adds InterceptHandler to route stdlib logs through Loguru; disables color when using JSON serialization; improves filter logic for extra context
src/fides/api/tasks/init.py	5/5	Adds signal handler to prevent Celery from overriding Loguru configuration

_{5 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

src/fides/api/util/logger.py

codecov · 2025-10-30T19:58:13Z

Codecov Report

❌ Patch coverage is 81.25000% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.30%. Comparing base (fa0ae77) to head (9d3eb20).
⚠️ Report is 10 commits behind head on main.

Files with missing lines	Patch %	Lines
src/fides/api/db/database.py	0.00%	4 Missing ⚠️
src/fides/api/util/logger.py	91.66%	2 Missing ⚠️

❌ Your patch status has failed because the patch coverage (81.25%) is below the target coverage (100.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #6891   +/-   ##
=======================================
  Coverage   87.30%   87.30%           
=======================================
  Files         524      524           
  Lines       34355    34382   +27     
  Branches     3956     3957    +1     
=======================================
+ Hits        29992    30017   +25     
- Misses       3503     3505    +2     
  Partials      860      860

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

greptile-apps

Greptile Overview

Greptile Summary

This PR improves logging consistency by intercepting standard library logs (from Celery, SQLAlchemy, etc.) and routing them through Loguru for unified formatting and serialization.

Key changes:

Added InterceptHandler class to capture stdlib logs and route through Loguru
Connected to Celery's setup_logging signal to prevent Celery from overriding logging config
Fixed colorization to be disabled when using JSON serialization
Improved filter functions to properly check for custom extra context
Enabled Python warnings capture through logging

Issues found:

Path logic in InterceptHandler.emit() uses forward-slash splitting that will fail on Windows, causing incorrect stack depth calculation for all intercepted logs

Confidence Score: 3/5

This PR is safe to merge on Linux/Unix systems but has a critical path handling bug for Windows environments
Score reflects a well-designed logging improvement with proper Celery integration, but the Windows path compatibility issue in the InterceptHandler could cause all logs to have incorrect caller information on Windows, affecting log quality and debugging capability
Pay close attention to src/fides/api/util/logger.py - the path logic needs to use os.path.dirname() instead of string splitting for cross-platform compatibility

Important Files Changed

File Analysis

Filename	Score	Overview
src/fides/api/util/logger.py	3/5	Added InterceptHandler to unify logging through Loguru and fixed JSON colorization; path logic has Windows compatibility issue

_{1 file reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

src/fides/api/util/logger.py

RobertKeyser · 2025-10-30T20:41:05Z

@greptileai

greptile-apps

Greptile Overview

Greptile Summary

This PR improves logging consistency by intercepting standard library logs (Celery, SQLAlchemy, etc.) and routing them through Loguru for uniform formatting and serialization.

Key changes:

Added InterceptHandler class to capture stdlib logging and redirect to Loguru
Added Celery signal handler to prevent Celery from overriding logging config
Fixed colorization to be disabled when JSON serialization is enabled
Replaced lambda filters with named functions for better code clarity
Changed print() statements to proper logger.info() calls

The implementation addresses two previously identified issues with the frame traversal and path logic that have been resolved with os.path.dirname() and os.sep.

Confidence Score: 4/5

This PR is safe to merge with low risk of regressions
The changes are well-tested with comprehensive unit tests covering the InterceptHandler functionality. The implementation follows best practices for log interception and the previous path logic issues have been addressed. Minor deduction for the potential impact on logging behavior across the entire application
No files require special attention

Important Files Changed

File Analysis

Filename	Score	Overview
src/fides/api/util/logger.py	4/5	Added InterceptHandler to capture stdlib logs and route through Loguru, improved colorize logic for JSON serialization, replaced lambda filters with named functions

_{2 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

vercel · 2025-10-31T16:42:50Z

Deployment failed with the following error:

You must set up Two-Factor Authentication before accessing this team.

View Documentation: https://vercel.com/docs/two-factor-authentication

galvana · 2025-11-07T06:20:08Z

CHANGELOG.md


 ### Changed
 - Updated filter modal in new privacy request screen to store filters as query params in url [#6818](https://github.com/ethyca/fides/pull/6818)
+- Updated logging configuration to intercept all standard library logs and route them through Loguru[#6891](https://github.com/ethyca/fides/pull/6891)


Suggested change

- Updated logging configuration to intercept all standard library logs and route them through Loguru[#6891](https://github.com/ethyca/fides/pull/6891)

- Updated logging configuration to intercept all standard library logs and route them through Loguru [#6891](https://github.com/ethyca/fides/pull/6891)

galvana

The changes look good, let's keep this one moving along. I went ahead and resolved the merge conflicts since they were caused by some recent changes I made to database.py

…log.info()

RobertKeyser self-assigned this Oct 30, 2025

RobertKeyser commented Oct 30, 2025

View reviewed changes

RobertKeyser marked this pull request as ready for review October 30, 2025 19:32

RobertKeyser requested a review from a team as a code owner October 30, 2025 19:32

RobertKeyser requested review from johnewart and removed request for a team October 30, 2025 19:32

greptile-apps bot reviewed Oct 30, 2025

View reviewed changes

src/fides/api/util/logger.py Show resolved Hide resolved

This comment was marked as resolved.

Sign in to view

greptile-apps bot reviewed Oct 30, 2025

View reviewed changes

src/fides/api/util/logger.py Outdated Show resolved Hide resolved

RobertKeyser requested review from galvana and tvandort October 30, 2025 20:19

RobertKeyser added a commit that referenced this pull request Oct 30, 2025

docs: CHANGELOG for #6891

51746d4

greptile-apps bot reviewed Oct 30, 2025

View reviewed changes

tvandort pushed a commit that referenced this pull request Oct 31, 2025

docs: CHANGELOG for #6891

8091509

tvandort force-pushed the rkeyser/fix-logging-serialization branch from de53e32 to 7d2d493 Compare October 31, 2025 21:17

galvana reviewed Nov 7, 2025

View reviewed changes

galvana approved these changes Nov 7, 2025

View reviewed changes

tvandort approved these changes Nov 7, 2025

View reviewed changes

RobertKeyser added 8 commits November 7, 2025 11:18

feat: intercept all logs to make sure they go through Loguru

2366b4f

refactor: replace print() statement with a proper loguru log.info()

d854c28

refactor: replace database.py print() statement with a proper loguru …

6369db6

…log.info()

fix: address mypy/pylint issues and fix logging logic

bdcbc7e

fix: more linting issues - remove unnecessary lamdba

7bb0564

fix: improve path logic

24b0d13

fix: black issue

09e94f3

fix: improve how we're dealing with directories

a988e7e

RobertKeyser added 3 commits November 7, 2025 11:20

docs: CHANGELOG for #6891

f394796

fix: failing test

e343307

refactor: run isort

9d3eb20

tvandort force-pushed the rkeyser/fix-logging-serialization branch from 1ab8ea4 to 9d3eb20 Compare November 7, 2025 16:20

tvandort added this pull request to the merge queue Nov 7, 2025

Merged via the queue into main with commit 9672d8d Nov 7, 2025
68 of 69 checks passed

tvandort deleted the rkeyser/fix-logging-serialization branch November 7, 2025 17:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Improve Logging Serialization #6891

Improve Logging Serialization #6891

RobertKeyser commented Oct 30, 2025 •

edited

Loading

Uh oh!

vercel bot commented Oct 30, 2025 •

edited

Loading

Uh oh!

RobertKeyser Oct 30, 2025

Uh oh!

greptile-apps bot left a comment

Uh oh!

Uh oh!

codecov bot commented Oct 30, 2025 •

edited

Loading

Uh oh!

This comment was marked as resolved.

greptile-apps bot left a comment

Uh oh!

Uh oh!

RobertKeyser commented Oct 30, 2025

Uh oh!

greptile-apps bot left a comment

Uh oh!

vercel bot commented Oct 31, 2025

Uh oh!

galvana Nov 7, 2025

Uh oh!

galvana left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	- Updated logging configuration to intercept all standard library logs and route them through Loguru[#6891](https://github.com/ethyca/fides/pull/6891)
	- Updated logging configuration to intercept all standard library logs and route them through Loguru [#6891](https://github.com/ethyca/fides/pull/6891)

Improve Logging Serialization #6891

Improve Logging Serialization #6891

Conversation

RobertKeyser commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description Of Changes

Code Changes

Steps to Confirm

Pre-Merge Checklist

Uh oh!

vercel bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RobertKeyser Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Greptile Overview

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Uh oh!

Uh oh!

codecov bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

This comment was marked as resolved.

greptile-apps bot left a comment

Choose a reason for hiding this comment

Greptile Overview

Greptile Summary

Confidence Score: 3/5

Important Files Changed

Uh oh!

Uh oh!

RobertKeyser commented Oct 30, 2025

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Greptile Overview

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Uh oh!

vercel bot commented Oct 31, 2025

Uh oh!

galvana Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

galvana left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

RobertKeyser commented Oct 30, 2025 •

edited

Loading

vercel bot commented Oct 30, 2025 •

edited

Loading

codecov bot commented Oct 30, 2025 •

edited

Loading