Skip to content

Fix spurious failures for macos packages#901

Open
psavery wants to merge 2 commits intomasterfrom
fix-macos-package-spurious-failures
Open

Fix spurious failures for macos packages#901
psavery wants to merge 2 commits intomasterfrom
fix-macos-package-spurious-failures

Conversation

@psavery
Copy link
Collaborator

@psavery psavery commented Feb 27, 2026

Overview

Sometimes, between fetching an index and downloading the actual package, there's an update to a package, which causes a checksum mismatch. This has been causing macos build failures.

Clearing the package cache before conda build helps prevent this failure from happening. We also add retry logic so that if the upstream race condition occurs during the actual build, it will hopefully succeed the next time.

I think it is mainly happening for mac because we include a cross-compile, and thus build 10 packages in one CI run. I think conda build caches the hashsum values early on in the process, and if the packages get updated upstream during the 10 builds, the hashsum no longer matches and it fails.

Affected Workflows

None

Documentation Changes

None

Sometimes, between fetching an index and downloading the actual
package, there's an update to a package, which causes a checksum
mismatch. This has been causing macos build failures.

Clearing the package cache before `conda build` helps prevent this
failure from happening. We also add retry logic so that if the
upstream race condition occurs during the actual build, it will
hopefully succeed the next time.

Signed-off-by: Patrick Avery <patrick.avery@kitware.com>
@psavery
Copy link
Collaborator Author

psavery commented Feb 27, 2026

I'm gonna try running the CI 3 times to verify the spurious error doesn't happen again, at least in this PR. It's running the first time now.

@codecov
Copy link

codecov bot commented Feb 27, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.18%. Comparing base (f40f031) to head (8123db2).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #901      +/-   ##
==========================================
- Coverage   69.41%   69.18%   -0.23%     
==========================================
  Files         148      148              
  Lines       22590    22590              
==========================================
- Hits        15680    15629      -51     
- Misses       6910     6961      +51     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@psavery psavery requested a review from bnmajor February 27, 2026 18:37
@psavery
Copy link
Collaborator Author

psavery commented Feb 27, 2026

Strangely, the code coverage is reduced, even though only CI code was modified 🙂

@psavery
Copy link
Collaborator Author

psavery commented Feb 27, 2026

First time passed. Rerunning.

@psavery
Copy link
Collaborator Author

psavery commented Feb 27, 2026

Second time passed. It succeeded during its second attempt (first attempt had the error).

This will *really* ensure it won't fail.

Signed-off-by: Patrick Avery <patrick.avery@kitware.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant