Skip to content

Comments

Fix tests to work with out of source builds.#1926

Open
Dimi1010 wants to merge 59 commits intoseladb:devfrom
Dimi1010:fix/out-of-source-tests
Open

Fix tests to work with out of source builds.#1926
Dimi1010 wants to merge 59 commits intoseladb:devfrom
Dimi1010:fix/out-of-source-tests

Conversation

@Dimi1010
Copy link
Collaborator

@Dimi1010 Dimi1010 commented Aug 16, 2025

Summary

This PR refactors the project to output the PCPP Test executables inside the default output directory and updates the CI build_and_test workflow to use the new location.

Rationale

The previous implementation used built the test executables within the source directories. The rationale for that was to allow usage of PcapExamples folder inside the test folder for samples. The approach however created several issues with modern build patterns such as multi-config generators like Visual Studio and Ninja Multi-Config. Those solutions rely partially on out of source builds, by having the output artifacts that are config specific be generated in config sub-folders (e.g. Debug, Release, etc).

The hardcoded output location of the executable broke functionalities such as the ability to have multiple configuration executables cached. For example switching to a Release config and building would overwrite a previously built Debug target.

The refactor updates the project to output all of its artifacts inside the build directory, keeping the source stable allowing more predictable behaviour when utilizing modern generators.

Changes

Test executable output location change

The test executable output location has been moved to the default location (e.g. $(buildDir)/Tests/Packet++Test/<Config>/Packet++Test.exe).

To keep compatibility the contents of PcapExamples are copied next to the executable. A possible future fix is to provide the ability for executable to load test resources from a custom location instead of relying on samples being inside the working directory.

Refactor of run_tests scripts

The CI run tests python scripts have been refactored to allow for variable locations for build directory and test executables.

The change also introduces a new Runner class that separates most of the logic into smaller functions instead of having a monolith main function, improving readability. Additional logging has also been added to help with runtime debugging.

Github Actions workflow update

The build_and_test workflow has been updated to utilize the refactored run_tests by passing the build directory of the project, and in some cases override on the default relative paths to the test executables.

The workflow has also been updated to utilize the working-directory parameter on action steps instead of using a cd ... command inside shell command of the run parameter.

@Dimi1010 Dimi1010 added tests build any build issue labels Aug 16, 2025
@seladb
Copy link
Owner

seladb commented Nov 15, 2025

@Dimi1010 should we close this PR?

@Dimi1010
Copy link
Collaborator Author

Sure, we can do it for now.

@Dimi1010 Dimi1010 closed this Nov 15, 2025
@Dimi1010 Dimi1010 reopened this Dec 11, 2025
@codecov
Copy link

codecov bot commented Dec 11, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.81%. Comparing base (bcc9ffd) to head (988ee88).

Additional details and impacted files
@@           Coverage Diff            @@
##              dev    #1926    +/-   ##
========================================
  Coverage   83.81%   83.81%            
========================================
  Files         313      313            
  Lines       55676    55669     -7     
  Branches    11606    11821   +215     
========================================
- Hits        46665    46660     -5     
+ Misses       7810     7807     -3     
- Partials     1201     1202     +1     
Flag Coverage Δ
alpine320 76.40% <ø> (+<0.01%) ⬆️
fedora42 76.12% <ø> (-0.03%) ⬇️
macos-14 81.96% <100.00%> (-0.01%) ⬇️
macos-15 81.96% <100.00%> (ø)
mingw32 70.35% <ø> (-0.02%) ⬇️
mingw64 70.35% <ø> (+0.07%) ⬆️
npcap ?
rhel94 75.78% <ø> (+0.03%) ⬆️
ubuntu2004 59.59% <0.00%> (ø)
ubuntu2004-zstd 59.70% <0.00%> (-0.03%) ⬇️
ubuntu2204 75.69% <ø> (+<0.01%) ⬆️
ubuntu2204-icpx 59.07% <ø> (ø)
ubuntu2404 76.07% <ø> (+<0.01%) ⬆️
ubuntu2404-arm64 76.06% <ø> (+0.02%) ⬆️
unittest 83.81% <100.00%> (+<0.01%) ⬆️
windows-2022 85.61% <ø> (+0.11%) ⬆️
windows-2025 85.65% <ø> (+0.13%) ⬆️
winpcap 85.65% <ø> (-0.09%) ⬇️
xdp 51.94% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Dimi1010 Dimi1010 marked this pull request as ready for review January 2, 2026 20:39
@Dimi1010 Dimi1010 requested a review from seladb as a code owner January 2, 2026 20:39
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto: are the changes related to this PR?

@seladb
Copy link
Owner

seladb commented Jan 10, 2026

@Dimi1010 the reason the coverage tests are failing is a new gcovr version that was released recently. I'm not entirely sure how to fix it, but for now we can pin to the latest version supported by all platforms as I did here

@Dimi1010 Dimi1010 requested a review from seladb February 8, 2026 14:50
Copy link
Owner

@seladb seladb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see one comment, overall LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build any build issue tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants