Skip to content

Conversation

Copy link

Copilot AI commented Dec 11, 2025

The shuffle filter was being applied based solely on the H5shuffle flag, without verifying compression was enabled. This added unnecessary overhead when H5compress=0 and H5szip=false.

Changes

  • EmpCylSL.cc: Changed condition from if (H5compress) to if (H5compress > 0 || H5szip) to support both Deflate and Szip compression modes
  • BiorthBasis.cc: Updated three compression sections:
    • Sample stats: Deflate-only with if (H5compress > 0)
    • Coefficients/covariance: Both compression types with if (H5compress > 0 || H5szip)
  • Added comments clarifying that shuffle is only applied when the outer condition guarantees compression is enabled

Behavior

Before:

if (H5compress) {
  if (H5shuffle) dcpl.add(HighFive::Shuffle());
  // Always applies shuffle when H5shuffle=true and H5compress!=0
}

After:

if (H5compress > 0 || H5szip) {
  // Only apply shuffle filter when compression is enabled (guaranteed by outer if)
  if (H5shuffle) dcpl.add(HighFive::Shuffle());
  // Shuffle only applies when Deflate (H5compress>0) or Szip is enabled
}

This prevents shuffle overhead when compression is disabled while allowing Szip to be used independently of the H5compress level setting.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 6 commits December 11, 2025 14:27
Co-authored-by: The9Cat <25960766+The9Cat@users.noreply.github.com>
…tion

Co-authored-by: The9Cat <25960766+The9Cat@users.noreply.github.com>
Co-authored-by: The9Cat <25960766+The9Cat@users.noreply.github.com>
Co-authored-by: The9Cat <25960766+The9Cat@users.noreply.github.com>
Co-authored-by: The9Cat <25960766+The9Cat@users.noreply.github.com>
Co-authored-by: The9Cat <25960766+The9Cat@users.noreply.github.com>
Copilot AI changed the title [WIP] Update HDF5 compression in EmpCylSL disk cache files Only apply HDF5 shuffle filter when compression is enabled Dec 11, 2025
Copilot AI requested a review from The9Cat December 11, 2025 14:37
@The9Cat The9Cat marked this pull request as ready for review December 11, 2025 14:40
@The9Cat The9Cat merged commit a4f1c09 into hdf5compress Dec 11, 2025
@The9Cat The9Cat deleted the copilot/sub-pr-183-yet-again branch December 11, 2025 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants