-
Notifications
You must be signed in to change notification settings - Fork 84
Improve Privacy Center Container Performance #6689
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve Privacy Center Container Performance #6689
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 2 Skipped Deployments
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Overview
Summary
This PR introduces significant performance improvements to the Privacy Center by implementing two key optimizations: in-memory caching of JavaScript bundles and HTTP `stale-while-revalidate` caching headers.The changes eliminate repeated file I/O operations by caching static JavaScript bundles (fides.js variants and GPP extension) in server memory after the first request. This replaces the previous behavior of reading files from disk on every request. The implementation includes proper error handling to prevent partial cache states and uses Promise.all for efficient parallel loading of multiple bundle variants.
Additionally, the PR adds HTTP stale-while-revalidate cache control headers, allowing clients to serve cached content immediately while fetching fresh versions in the background. For fides.js, this is configurable via the FIDES_PRIVACY_CENTER__FIDES_JS_STALE_WHILE_REVALIDATE_SECONDS environment variable (defaulting to 24 hours), while the GPP extension uses a fixed 24-hour value following the existing pattern.
The changes maintain backward compatibility and follow established patterns in the codebase. The bundle selection logic remains unchanged - only the underlying file access mechanism is optimized. These optimizations are particularly beneficial for static JavaScript resources that don't change frequently but are critical for initial page loads.
Important Files Changed
Changed Files
| Filename | Score | Overview |
|---|---|---|
clients/privacy-center/app/server-utils/PrivacyCenterSettings.ts |
5/5 | Added TypeScript interface definition for the new stale-while-revalidate configuration option |
clients/privacy-center/app/server-environment.ts |
5/5 | Added server-side configuration loading for the new stale-while-revalidate setting |
clients/privacy-center/app/server-utils/loadEnvironmentVariables.ts |
4/5 | Added environment variable parsing and default value constants for stale-while-revalidate |
clients/privacy-center/pages/api/fides-js.ts |
4/5 | Implemented in-memory bundle caching and stale-while-revalidate headers for fides.js variants |
clients/privacy-center/pages/api/fides-ext-gpp-js.ts |
4/5 | Added in-memory caching and stale-while-revalidate headers for GPP extension bundle |
Confidence score: 4/5
- This PR is safe to merge with low risk as it implements well-established caching patterns
- Score reflects solid implementation of performance optimizations with proper error handling and backward compatibility
- Pay attention to the API endpoint files to ensure caching behavior works correctly in production environments
Sequence Diagram
sequenceDiagram
participant User
participant Browser
participant PrivacyCenter
participant FileSystem
participant FidesAPI
participant CDN/Cache
User->>Browser: "Visit website with fides.js"
Browser->>PrivacyCenter: "GET /api/fides-js.ts"
PrivacyCenter->>PrivacyCenter: "loadServerSettings()"
PrivacyCenter->>PrivacyCenter: "loadStaticBundles()"
alt First request or bundles not cached
PrivacyCenter->>FileSystem: "Read fides.js bundles"
FileSystem-->>PrivacyCenter: "Bundle contents"
PrivacyCenter->>PrivacyCenter: "Cache bundles in memory"
end
PrivacyCenter->>PrivacyCenter: "getPrivacyCenterEnvironmentCached()"
PrivacyCenter->>PrivacyCenter: "lookupGeolocation()"
alt Prefetch enabled and geolocation available
PrivacyCenter->>FidesAPI: "fetchExperience() with retry"
FidesAPI-->>PrivacyCenter: "Privacy experience data"
end
alt Custom CSS refresh needed
PrivacyCenter->>FidesAPI: "GET /plus/custom-asset/custom-fides.css"
FidesAPI-->>PrivacyCenter: "Custom CSS content"
PrivacyCenter->>PrivacyCenter: "Cache CSS with TTL"
end
PrivacyCenter->>PrivacyCenter: "Build FidesConfig JSON"
PrivacyCenter->>PrivacyCenter: "Bundle script with config"
PrivacyCenter-->>Browser: "Complete fides.js with stale-while-revalidate headers"
Browser->>CDN/Cache: "Cache with max-age and stale-while-revalidate"
Browser->>Browser: "Execute fides.js bundle"
Browser-->>User: "Privacy experience rendered"
Note over User,CDN/Cache: Subsequent requests served from cache with stale-while-revalidate behavior
5 files reviewed, 1 comment
This comment was marked as resolved.
This comment was marked as resolved.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Overview
Summary
This review covers only the changes made since the last review, not the entire PR. The developer has implemented performance improvements for the Privacy Center container by introducing in-memory caching for static JavaScript bundles and adding advanced HTTP cache-control headers. The changes address a previous style issue where `bundlesLoaded = false` was redundantly set in error handling.The core modifications include:
-
In-memory caching system: The fides.js API endpoint now uses
loadStaticBundles()to cache all four bundle variants (fides.js, fides-tcf.js, fides-headless.js, fides-ext-gpp.js) in module-level variables after the first request, eliminating repeated file I/O operations. -
Advanced cache-control headers: Both the main fides.js and GPP extension endpoints now support
stale-while-revalidateandstale-if-errordirectives. These allow browsers to serve cached content immediately while fetching fresh content in the background, and serve stale content when the origin server fails. -
Configurable stale serving: A new environment variable
FIDES_PRIVACY_CENTER__FIDES_JS_SERVE_STALE_SECONDS(defaulting to 24 hours) controls the stale content behavior for fides.js, following the established pattern used for other cache settings.
The implementation follows existing patterns in the codebase and integrates cleanly with the Privacy Center's server environment configuration system. The changes are focused on static asset delivery optimization without altering core functionality.
Important Files Changed
Changed Files
| Filename | Score | Overview |
|---|---|---|
| clients/privacy-center/pages/api/fides-js.ts | 5/5 | Implemented in-memory caching for static bundles and added configurable stale cache-control headers |
| clients/privacy-center/pages/api/fides-ext-gpp-js.ts | 4/5 | Added in-memory caching and stale-while-revalidate headers for GPP extension bundle |
| clients/privacy-center/app/server-environment.ts | 5/5 | Added FIDES_JS_SERVE_STALE_SECONDS environment variable to server settings configuration |
| clients/privacy-center/app/server-utils/PrivacyCenterSettings.ts | 5/5 | Added FIDES_JS_SERVE_STALE_SECONDS setting to the interface with proper documentation |
| clients/privacy-center/app/server-utils/loadEnvironmentVariables.ts | 5/5 | Implemented environment variable loading for stale cache configuration with sensible defaults |
Confidence score: 4/5
- This PR is safe to merge with minimal risk as it implements well-established caching patterns
- Score reflects solid implementation of performance optimizations using proven HTTP caching strategies
- Pay close attention to clients/privacy-center/pages/api/fides-ext-gpp-js.ts for potential edge cases in the caching logic
5 files reviewed, no comments
| let cachedFidesJs: string = ""; | ||
| let cachedFidesTcfJs: string = ""; | ||
| let cachedFidesHeadlessJs: string = ""; | ||
| let cachedFidesGppJs: string = ""; | ||
| let bundlesLoaded: boolean = false; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Quick question about this, do we know if this changes the over all memory overhead needed to run the privacy center? I can imagine this wouldn't actually end up being different than what we had previously and maybe even lowers the memory usage on the server but I'd be keen to know if that assumption is right.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My educated guess is that it will result in a minimal increase in the memory overhead. Through some back of the napkin math, I'd say an increase of ~0.8GiB or so.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh also, this should have a similarly minor CPU load and disk read decrease since we'll not be reading from disk each time.
This should save us some IO time for requests that make it down to the privacy center server
76d304b to
21e25ae
Compare
|
Deployment failed with the following error: View Documentation: https://vercel.com/docs/two-factor-authentication |
Closes []
Description Of Changes
This PR has some performance improvements for the Privacy Center. These include avoiding per-request file reads and adds
stale-while-revalidateandstale-if-errorcache-control header values to improve performance and reliability.stale-while-revalidateallows a cached (stale) version of the page to be served immediately while a fresh version is fetched in the background, andstale-if-errorserves a stale version if the origin server returns an error, preventing users from seeing a broken page.Code Changes
stale-while-revalidateandstale-if-errorresponse header, for fides.jsFIDES_PRIVACY_CENTER__FIDES_JS_SERVE_STALE_SECONDS, defaults to 24 hoursstale-while-revalidateandstale-if-errorheader for gpp (not configurable, following existing GPP_JS_MAX_AGE_SECONDS pattern)Steps to Confirm
npm run dev-pcstale-while-revalidateI'd expect the default header to be:
max-age=3600, stale-while-revalidate=86400, stale-if-error=86400, publicPre-Merge Checklist
CHANGELOG.mdupdatedmaindowngrade()migration is correct and works