Skip to content

Conversation

@RobertKeyser
Copy link
Contributor

@RobertKeyser RobertKeyser commented Oct 1, 2025

Closes []

Description Of Changes

This PR has some performance improvements for the Privacy Center. These include avoiding per-request file reads and adds stale-while-revalidate and stale-if-error cache-control header values to improve performance and reliability. stale-while-revalidate allows a cached (stale) version of the page to be served immediately while a fresh version is fetched in the background, and stale-if-error serves a stale version if the origin server returns an error, preventing users from seeing a broken page.

Code Changes

  • Add stale-while-revalidate and stale-if-error response header, for fides.js
    • Configurable with the environment variable FIDES_PRIVACY_CENTER__FIDES_JS_SERVE_STALE_SECONDS, defaults to 24 hours
    • Set to 0 to disable this headers.
  • Add stale-while-revalidate and stale-if-error header for gpp (not configurable, following existing GPP_JS_MAX_AGE_SECONDS pattern)

Steps to Confirm

  1. Run npm run dev-pc
  2. browse to fides.js, make sure it still works
  3. check response headers for stale-while-revalidate
  4. Try out the env var

I'd expect the default header to be:
max-age=3600, stale-while-revalidate=86400, stale-if-error=86400, public

Pre-Merge Checklist

  • Issue requirements met
  • All CI pipelines succeeded
  • CHANGELOG.md updated
    • Add a db-migration This indicates that a change includes a database migration label to the entry if your change includes a DB migration
    • Add a high-risk This issue suggests changes that have a high-probability of breaking existing code label to the entry if your change includes a high-risk change (i.e. potential for performance impact or unexpected regression) that should be flagged
    • Updates unreleased work already in Changelog, no new entry necessary
  • Followup issues:
    • Followup issues created
    • No followup issues
  • Database migrations:
    • Ensure that your downrev is up to date with the latest revision on main
    • Ensure that your downgrade() migration is correct and works
      • If a downgrade migration is not possible for this change, please call this out in the PR description!
    • No migrations
  • Documentation:
    • Documentation complete, PR opened in fidesdocs
    • Documentation issue created in fidesdocs
    • If there are any new client scopes created as part of the pull request, remember to update public-facing documentation that references our scope registry
    • No documentation updates required

@vercel
Copy link

vercel bot commented Oct 1, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
Project Deployment Preview Comments Updated (UTC)
fides-plus-nightly Ignored Ignored Preview Oct 31, 2025 9:14pm
fides-privacy-center Ignored Ignored Oct 31, 2025 9:14pm

@RobertKeyser RobertKeyser marked this pull request as ready for review October 1, 2025 21:30
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Overview

Summary

This PR introduces significant performance improvements to the Privacy Center by implementing two key optimizations: in-memory caching of JavaScript bundles and HTTP `stale-while-revalidate` caching headers.

The changes eliminate repeated file I/O operations by caching static JavaScript bundles (fides.js variants and GPP extension) in server memory after the first request. This replaces the previous behavior of reading files from disk on every request. The implementation includes proper error handling to prevent partial cache states and uses Promise.all for efficient parallel loading of multiple bundle variants.

Additionally, the PR adds HTTP stale-while-revalidate cache control headers, allowing clients to serve cached content immediately while fetching fresh versions in the background. For fides.js, this is configurable via the FIDES_PRIVACY_CENTER__FIDES_JS_STALE_WHILE_REVALIDATE_SECONDS environment variable (defaulting to 24 hours), while the GPP extension uses a fixed 24-hour value following the existing pattern.

The changes maintain backward compatibility and follow established patterns in the codebase. The bundle selection logic remains unchanged - only the underlying file access mechanism is optimized. These optimizations are particularly beneficial for static JavaScript resources that don't change frequently but are critical for initial page loads.

Important Files Changed

Changed Files
Filename Score Overview
clients/privacy-center/app/server-utils/PrivacyCenterSettings.ts 5/5 Added TypeScript interface definition for the new stale-while-revalidate configuration option
clients/privacy-center/app/server-environment.ts 5/5 Added server-side configuration loading for the new stale-while-revalidate setting
clients/privacy-center/app/server-utils/loadEnvironmentVariables.ts 4/5 Added environment variable parsing and default value constants for stale-while-revalidate
clients/privacy-center/pages/api/fides-js.ts 4/5 Implemented in-memory bundle caching and stale-while-revalidate headers for fides.js variants
clients/privacy-center/pages/api/fides-ext-gpp-js.ts 4/5 Added in-memory caching and stale-while-revalidate headers for GPP extension bundle

Confidence score: 4/5

  • This PR is safe to merge with low risk as it implements well-established caching patterns
  • Score reflects solid implementation of performance optimizations with proper error handling and backward compatibility
  • Pay attention to the API endpoint files to ensure caching behavior works correctly in production environments

Sequence Diagram

sequenceDiagram
    participant User
    participant Browser
    participant PrivacyCenter
    participant FileSystem
    participant FidesAPI
    participant CDN/Cache

    User->>Browser: "Visit website with fides.js"
    Browser->>PrivacyCenter: "GET /api/fides-js.ts"
    
    PrivacyCenter->>PrivacyCenter: "loadServerSettings()"
    PrivacyCenter->>PrivacyCenter: "loadStaticBundles()"
    
    alt First request or bundles not cached
        PrivacyCenter->>FileSystem: "Read fides.js bundles"
        FileSystem-->>PrivacyCenter: "Bundle contents"
        PrivacyCenter->>PrivacyCenter: "Cache bundles in memory"
    end
    
    PrivacyCenter->>PrivacyCenter: "getPrivacyCenterEnvironmentCached()"
    PrivacyCenter->>PrivacyCenter: "lookupGeolocation()"
    
    alt Prefetch enabled and geolocation available
        PrivacyCenter->>FidesAPI: "fetchExperience() with retry"
        FidesAPI-->>PrivacyCenter: "Privacy experience data"
    end
    
    alt Custom CSS refresh needed
        PrivacyCenter->>FidesAPI: "GET /plus/custom-asset/custom-fides.css"
        FidesAPI-->>PrivacyCenter: "Custom CSS content"
        PrivacyCenter->>PrivacyCenter: "Cache CSS with TTL"
    end
    
    PrivacyCenter->>PrivacyCenter: "Build FidesConfig JSON"
    PrivacyCenter->>PrivacyCenter: "Bundle script with config"
    PrivacyCenter-->>Browser: "Complete fides.js with stale-while-revalidate headers"
    
    Browser->>CDN/Cache: "Cache with max-age and stale-while-revalidate"
    Browser->>Browser: "Execute fides.js bundle"
    Browser-->>User: "Privacy experience rendered"

    Note over User,CDN/Cache: Subsequent requests served from cache with stale-while-revalidate behavior
Loading

5 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

@RobertKeyser

This comment was marked as resolved.

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Overview

Summary

This review covers only the changes made since the last review, not the entire PR. The developer has implemented performance improvements for the Privacy Center container by introducing in-memory caching for static JavaScript bundles and adding advanced HTTP cache-control headers. The changes address a previous style issue where `bundlesLoaded = false` was redundantly set in error handling.

The core modifications include:

  1. In-memory caching system: The fides.js API endpoint now uses loadStaticBundles() to cache all four bundle variants (fides.js, fides-tcf.js, fides-headless.js, fides-ext-gpp.js) in module-level variables after the first request, eliminating repeated file I/O operations.

  2. Advanced cache-control headers: Both the main fides.js and GPP extension endpoints now support stale-while-revalidate and stale-if-error directives. These allow browsers to serve cached content immediately while fetching fresh content in the background, and serve stale content when the origin server fails.

  3. Configurable stale serving: A new environment variable FIDES_PRIVACY_CENTER__FIDES_JS_SERVE_STALE_SECONDS (defaulting to 24 hours) controls the stale content behavior for fides.js, following the established pattern used for other cache settings.

The implementation follows existing patterns in the codebase and integrates cleanly with the Privacy Center's server environment configuration system. The changes are focused on static asset delivery optimization without altering core functionality.

Important Files Changed

Changed Files
Filename Score Overview
clients/privacy-center/pages/api/fides-js.ts 5/5 Implemented in-memory caching for static bundles and added configurable stale cache-control headers
clients/privacy-center/pages/api/fides-ext-gpp-js.ts 4/5 Added in-memory caching and stale-while-revalidate headers for GPP extension bundle
clients/privacy-center/app/server-environment.ts 5/5 Added FIDES_JS_SERVE_STALE_SECONDS environment variable to server settings configuration
clients/privacy-center/app/server-utils/PrivacyCenterSettings.ts 5/5 Added FIDES_JS_SERVE_STALE_SECONDS setting to the interface with proper documentation
clients/privacy-center/app/server-utils/loadEnvironmentVariables.ts 5/5 Implemented environment variable loading for stale cache configuration with sensible defaults

Confidence score: 4/5

  • This PR is safe to merge with minimal risk as it implements well-established caching patterns
  • Score reflects solid implementation of performance optimizations using proven HTTP caching strategies
  • Pay close attention to clients/privacy-center/pages/api/fides-ext-gpp-js.ts for potential edge cases in the caching logic

5 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

RobertKeyser added a commit that referenced this pull request Oct 1, 2025
@RobertKeyser RobertKeyser self-assigned this Oct 2, 2025
Comment on lines +45 to +49
let cachedFidesJs: string = "";
let cachedFidesTcfJs: string = "";
let cachedFidesHeadlessJs: string = "";
let cachedFidesGppJs: string = "";
let bundlesLoaded: boolean = false;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick question about this, do we know if this changes the over all memory overhead needed to run the privacy center? I can imagine this wouldn't actually end up being different than what we had previously and maybe even lowers the memory usage on the server but I'd be keen to know if that assumption is right.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My educated guess is that it will result in a minimal increase in the memory overhead. Through some back of the napkin math, I'd say an increase of ~0.8GiB or so.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh also, this should have a similarly minor CPU load and disk read decrease since we'll not be reading from disk each time.

@tvandort tvandort force-pushed the rkeyser/privacy-center-performance-improvement branch from 76d304b to 21e25ae Compare October 31, 2025 21:12
@vercel
Copy link

vercel bot commented Oct 31, 2025

Deployment failed with the following error:

You must set up Two-Factor Authentication before accessing this team.

View Documentation: https://vercel.com/docs/two-factor-authentication

@tvandort tvandort enabled auto-merge October 31, 2025 21:16
@tvandort tvandort added this pull request to the merge queue Oct 31, 2025
Merged via the queue into main with commit 51ad29e Oct 31, 2025
42 checks passed
@tvandort tvandort deleted the rkeyser/privacy-center-performance-improvement branch October 31, 2025 21:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants