make coreboot 4.11 platforms buildable under recent OSea (CircleCI builds on debian-11 here) #1106
Conversation
|
It was coincidental that it worked for debian-10, since locally deployed make was the good one per OS deployed versions (make 4.2.1) so the Makefile test was successful: Also to be noted that under debian-10, gawk was not installed by default, the Makefile test depending on something else then gawk if not found... On debian-10, we consequently see that gawk is reported to be in a "bad version" per global Makefile, but was still not compiled: https://circleci.com/api/v1.1/project/github/tlaurion/heads/3735/output/103/0?file=true&allocation-id=61fbece2b47315079ce0ff17-0-build%2F30054AE1 prior of musl-cross-make target. Also to be noted that the locally built make and gawk were not properly being propagated for make calls for all other modules. awk was available on the system. Basically, Heads was still using host deployed versions. |
tlaurion
force-pushed
the
CircleCI_retry_coreboot-411_under_debian-11
branch
from
7232f53 to
d5e68a1
Compare
|
Rebased on master to only include changes related to support debian-11 |
|
Now, build fails at https://app.circleci.com/pipelines/github/tlaurion/heads/961/workflows/51bc0444-e29d-42f8-804e-ee3dabd0ea8c/jobs/3902/parallel-runs/0/steps/0-103 : I am redoing build on top of debian-10 with the modified Makefile to see if it breaks here: https://app.circleci.com/pipelines/github/tlaurion/heads/962/workflows/8ed9bcc0-4539-4bed-88e2-6296ee203692 |
|
Builds successfully on top of debian-10 with same modified Makefile worked here: https://app.circleci.com/pipelines/github/tlaurion/heads/962/workflows/7af97058-b209-4e3d-8a4f-2250cb1fd25c and commit So now i'm even more puzzled now. We are now sure here vs debian-10 that:
Then why:
On screen log trace on debian-10: On screen log trace on debian-11: Someone?! Concurrence is the problem. But why!?! |
|
The next run fixes it magically. My eyes do not see what needs to be fixed under coreboot Makefile, Makefile.inc to fix that race condition... @MrChromebox? Same applies in current master for build.h So the parts generating build.h and assembly.inc are fragile in coreboot 4.11. What fixed it in later coreboot versions? |
|
Seems like https://review.coreboot.org/c/coreboot/+/42883 might fix assembly.inc race condition Edit: yesssssss |
|
And that for build.h race condition https://review.coreboot.org/c/coreboot/+/11904 Will test both... Edit: unfortunately, this specific upstream patch cannot be applied directly. Edit2: not needed. |
|
|
- Add kgpe-d16 patch to remove HID for PCI devices (successful build on top of linuxboot#1101 and linuxboot#1012 per https://app.circleci.com/pipelines/github/tlaurion/heads/937/workflows/de49bea0-3f58-4a91-8891-87622f5a0eed) - CircleCI modified to build for coreboot 4.11 kgpe-d16_workstation on top of 4.15 passed workspace - CircleCI modified so that we still archive all the logs in artifacts for the current build even if failing. We now exit 1 after having archived all the log files under build/ - Add xx30 vbios extract scripts to test. Expecting musl-cross target to fail since make and gawk aren't built - CircleCI: gawk was not installed in apt statements under Debian. Installing - Makefile: seperate and fix local make and gawk building pror of using. Otherwise, impossible to build musl-cross target seperatly. - Also give some debugging info at start of Heads builds to tell which local gawk and make are used, also telling which make call will be propagated in the rest of the builds - Fix gawk version checking, reporting bad version even if 4.2.1 as expected on debian-10 (debian-10 OS deploys gawk and make in version 4.2.1) - CircleCI: Changing musl-cross taget to bootstrap (gawk+make) and musl-cross-make (bootstrap_musl-cross-make) for clarity
…ng present and causing race condition in parallelized builds with high number of cores
tlaurion
force-pushed
the
CircleCI_retry_coreboot-411_under_debian-11
branch
from
a60fb21 to
510ce2b
Compare
|
Good news. Builds are now stable on debian-11 for coreboot 4.11 boards since bd0d080 and do not need reruns when there is concurrency problems (at least for 36 cores as under CircleCI) since local gawk and make are now actually properly propagated down the Heads buildsystem. Notice at beginning of builds on debian-11+ : Also notice that on debian-10, there is no gawk versioning mismatch anymore when 4.2.1 is expected (and found per OS deployment). On debian-10, both make and gawk OS deployed were in version 4.2.1. @MrChromebox a clean build, just to make sure, is happening under latest commit of this PR. Should be ready to merge, while your eyes on Makefile changes would still be nice. |
tlaurion
changed the title
boot tested successfully on L1UM |
|
CircleCI still hits a race condition for coreboot 4.11, which breaks CircleCI builds, even more consistently with #1178.... I cannot wrap my head around it.... end of consistently failing build log, enven when trying to trace issue from connecting to CircleCI through SSH and rebuilding now. With #1178, we are removing local make 4.2.1 locally being built and having all builds depend on that make 4.2.1 for everything being built.... That would be desirable for reasons explained under #1178. |
2 participants
This is WIP.In an attempt to investigate why car.data is not build under coreboot 4.11 builds under debian-11, I've decided to check what is going on in Heads' build system, which at the end of the day is supposed to do the right thing until we have something better to use.
To have reproducible builds, we are supposed to have fixed versions of some tools, prior of building anything in the hope of producing reproducible output:
If initrd is not reproducible, then check individual modules output under hashes.txt. Check logs for configure output of faulty module. Diffscope. Patch. Redo.
I first noticed that local gawk was only built if make was also in wrong local version, and then noticed that most of the make calls to build all modules were actually not using the locally built make version but the host deployed version, from both configure logs and build logs under build/logs/*.
This is an attempt to correct the situation.
My make fu not being optimal, please review and comment, and/or propose another PR with proper fixes. Also.... Please educate me if i'm doing stupid things when compared to original Makefile (which was obviously also doing stupid things.... as you can see by my attempt of improving it).
The DATE fixation Makefile statement was embedded under make version checking under Makefile as well. This is also fixed per current PR.
Still TODO: We lost rom hash output under hashes.txt. Not sure since when and why, yet. Things got broken when gawk local build was added, or around that time.