ENH: Replace the shell wheel build system with Python (supersedes #302) - #309
Draft
hjmjohnson wants to merge 12 commits into
Draft
hjmjohnson wants to merge 12 commits into
hjmjohnson wants to merge 12 commits into
Conversation
Co-Authored-By: Cavan Riley <cavan-riley@uiowa.edu>
Co-Authored-By: Cavan Riley <cavan-riley@uiowa.edu>
Co-Authored-By: Cavan Riley <cavan-riley@uiowa.edu>
Co-Authored-By: Cavan Riley <cavan-riley@uiowa.edu>
Co-Authored-By: Cavan Riley <cavan-riley@uiowa.edu>
Co-Authored-By: Cavan Riley <cavan-riley@uiowa.edu>
Co-Authored-By: Cavan Riley <cavan-riley@uiowa.edu>
Co-Authored-By: Cavan Riley <cavan-riley@uiowa.edu>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the shell-based wheel build system with a Python one built around
build_wheels.py, aTargetPlatformvalue object, and a platform-builderhierarchy, and adopts pixi and pre-commit. Supersedes #302, which was built
from the same work before testing on three platforms uncovered sixteen
defects.
Two changes narrow supported platforms. Both are deliberate and need a
maintainer decision, not just a code review:
auditwheel repairruns with
--only-plat --plat manylinux_2_28_<arch>on botharchitectures. Wheels stop installing on glibc 2.27 systems
(Ubuntu 18.04, EOL April 2023).
orphaned — every M1 and later runs Sonoma — but macOS 11 through 13 users
lose binary wheels.
Windows is unbuilt. It is the one platform with no test coverage here.
Two Windows defects are fixed in this branch but never exercised, so a
Windows run is needed before release.
Verification — what was actually built and checked
Every platform below was rebuilt from the committed tip of this branch
(
d2ec481), after all sixteen fixes, producing sevencp311-abi3wheelseach. The remote builder asserts the expected commit before building, so
these are not results from a stale checkout.
cp311-abi3-manylinux_2_28_x86_64cp311-abi3-manylinux_2_28_aarch64cp311-abi3-macosx_14_0_arm64Checked on each:
auditwheelrepair accounting (6 started, 6 written), allseven build steps reaching
done, and — on all three platforms — an importtest that first uninstalled the seven previously installed
itk*packagesand then reported
Successfully installed, rather than theRequirement already satisfiedthat would mean it had validated stalepackages instead of the wheels just built.
88 unit tests.
pre-commit run --all-filesclean.Why the wheel fixup prepends the build environment's lib directory
One line in the Linux fixup will look arbitrary without this.
LD_LIBRARY_PATHfor the repair step puts the build environment's ownlibfirst. Without it, pixi-provided binaries resolve against the container's
system libraries and
patchelffails to start withGLIBCXX_3.4.29' not found: the manylinux_2_28 image supplies at most3.4.25, the build environment3.4.36.auditwheelthen cannot constructits ELF patcher and every repair fails.
The ordering is the point, and the obvious alternative does not work:
dropping
/usr/lib64instead was tested and still fails, because the binaryhas no effective
RUNPATHback to the environment and falls through to/lib64.aarch64 is insensitive to the ordering because its
patchelfpredates thesymbols in question — so an x86_64-only failure here does not imply an
x86_64-only cause.
Why macOS 14.0, and why it is set in three places
ITK 5.4.x deliberately set 11.0:
itk-core5.4.7 was built on SDK 15.2 andcarries
minos 11.0. The published ITK 6 betas shipmacosx_15_0_arm64with
minostracking the build SDK exactly, which is the signature of nodeployment target being set at all — so upstream's 15.0 looks inherited from
its runner rather than chosen, and excludes macOS 14 users for no technical
reason.
14.0 is the lowest floor with no end-of-life macOS release below it, and it
orphans no Apple Silicon hardware.
Setting
MACOSX_DEPLOYMENT_TARGETalone is not sufficient.sysconfig.get_platform()reports the interpreter's build target andignores that variable, so the compile would produce
minos 14.0binariestagged
macosx_11_0— installable on macOS 11 through 13 and failing indyldat import._PYTHON_HOST_PLATFORMis derived from the same value andis what actually moves the tag. A test binds the three copies of the version
together so they cannot drift.
Known limitations a reviewer should weigh
make previously silent failures raise. Those paths execute only when a
command fails, so no green build exercises them. They are covered by tests
that fail against the pre-fix tree, but proving them live needs deliberate
failure injection — a corrupted wheel before repair, a bad ITK tag, a
failing
zstd.git bisect run pytestneedsrc=5treated as success. The firstcommit adopts pixi and pre-commit and introduces no symbol any test
exercises, so pytest reports "no tests ran" there. Every other commit
collects and passes.
manylinux_2_34remains reachable in the platform table as headroom,although the shipping decision is manylinux_2_28. It is commented as such
rather than removed.
Commit structure and dependency pinning
Twelve commits, each independently testable, ordered so the branch bisects.
Remote-module wheels are pinned to
itk >= 6.0, < 7. Note that an ITKrelease candidate does not satisfy that constraint under default pip rules;
installing against an RC needs
--pre.