scratch #1162
Draft
scratch #1162
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The goal of this series of commits is to migrate functionality from dpdk-sys (and give it a much needed refactor in the process). More specifically, this series of commits is intended to get the project to the point where you can do a sterile build of DPDK with the correct set of CFLAGS/LDFLAGS in both the debug and release profiles. Forming that build into a sysroot which can actually replace dpdk-sys will be left as future work. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
We begin the construction of our sysroot with a helper function designed to inject environment variables into a nixpkgs provided stdenv. This will (eventually) allow us to customize the flags used to compile each package needed for our sysroot. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
This commit introduces a nixpkgs provided stdenv based on the LLVM toolchain. We explicitly forswear GCC for this sysroot on the grounds that rustc is based on LLVM. Given that we want LTO to work properly, we need to ensure that all of our (modest list of) dataplane dependencies are built with LLVM. In a future commit we will add infrastructure to ensure that we are also always using the same version of LLVM used by our selected version of rustc. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
For reasons which make no sense to me, llvmPackages.stdenv does not seem to add LLVM's bintools or linker to the nativeBuildInputs by default. If you attempt to pass `-fuse-ld=lld` to the compiler without explicitly adding them to the build time dependency list then you fail to link. The fix is easy: inject those packages as dependencies. Note that I deliberately don't use `pkgs.lld` as that version of lld is not wrapped by nix and will not ensure correct settings for the rpath in generated elf files. llvmPackages.lld is the version we need here, as that version of lld is wrapped. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
It is neat that nixpkgs attempts to run unit tests for compiled packages, but in my experience those tests spuriously fail on the regular. The cause of these failures is almost uniformly due to insufficient permissions in the build environment. I will leave the testing of packages to the developers and package maintainers and simply disable those tests in our overlay. This is especially important in that we will (in the future) cross compile for aarch64 and our build machine would be unable to run the tests anyway. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Invoke or previously built helper function to pass user supplied flags into our new llvm stdenv. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Cook a quick helper function to override the stdenv of select packages with our static + llvm + user supplied flags stdenv. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
See comment for my reasoning. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
See comment for my reasoning. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
See comment for my reasoning. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
This is an indirect dependency of DPDK, so we want to customize its build. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
This is an indirect dependency of DPDK, so we want to customize its build. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
This is an dependency of DPDK, so we want to customize its build. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
This is an dependency of DPDK, so we want to customize its build. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
This is an dependency of DPDK, so we want to customize its build. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
This is a handy tool for "pinning" our external dependencies to known versions with fixed cryptographic hashes. I tried to make this work with nix flakes, but I had an extremely difficult time making cross compile work, and I frankly don't understand what flakes does that npins doesn't do objectively better. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
We want to have exact control over our version of rdma-core. We pin the version to use our fork of rdma-core which contains a minor patch sequence to help LTO compiles. This pin was added via ``` npins add github githedgehog rdma-core -b fix-lto-60.0 ``` Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Now that we have our pinned version of nixpkgs and rdma-core set up we can pipe those sources into our build system for consumption by the rest of our overlay. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Use the pinned version of nixpkgs. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
We currently only have one overlay in play, but I expect that to change when I add additional logic for FRR. Regardless, there is little cost in making the set of selected overlays flexible. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
These pinned sources need to be consumed by the dataplane overlay in order to pin our external dependencies, rdma-core and (soon) dpdk. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Use the newly piped in value of sources to fix rdma-core to our chosen version. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
DPDK recently released a new LTS version (25.11). Given that we are reworking our build system anyway, there is little reason to stay on the (now) outdated 25.07 version. This process will (in a future commit series) require regenerating our rust bindings anyway. While that is a relatively easy task, I can't see any real reason to do it twice. This pin was generated with the command ``` npins add github DPDK dpdk -b 25.11 ``` Signed-off-by: Daniel Noland <daniel@githedgehog.com>
This commit introduces a build recipe for DPDK optimized for our use case. This is largely just transferred over from dpdk-sys. The primary reason to focus on minimizing DPDK is to make it as easy as possible to bind DPDK to rust. In particular, enabling all of the drivers significantly complicates the binding process and significantly increases build times. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Call dpdk package from this repo rather than using the upstream build instructions (which are somewhat unfriendly to bindgen). Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Now that our overlay is mostly ready, we can re-export our overlay plus nixpkgs from default.nix. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Now that our overlay is mostly cooked, we can inject it into our pinned version of nixpkgs. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
We don't yet have the machinery to generate our compile flags, but we can pre-emptively wire up the system which will feed in those flags into our overlay. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Now that we have our overlay in a place where it can accept flags we can start constructing machinery to generate those flags. We start with the "common" profile. This profile consists of flags we expect to pass to each and every build environment on all target platforms. Flags should be added to this profile conservatively, as they apply globally. Do not add performance compromising flags here. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
This commit constructs the "debug" profile. This profile should only be activated in environments where performance is of much less concern than debugging. Signed-off-by: Daniel Noland <daniel@githedgehog.com>
daniel-noland
force-pushed
the
refactor-flags2
branch
from
d79332d to
c0292ed
Compare
daniel-noland
force-pushed
the
refactor-flags2
branch
from
79f1f28 to
47b0cf9
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.