Skip to content

ENH: Replace the shell wheel build system with Python (supersedes #302) - #309

Draft
hjmjohnson wants to merge 12 commits into
InsightSoftwareConsortium:mainfrom
BRAINSia:itk6-build-system-v2
Draft

hjmjohnson wants to merge 12 commits into
InsightSoftwareConsortium:mainfrom
BRAINSia:itk6-build-system-v2

Conversation

@hjmjohnson

@hjmjohnson hjmjohnson commented Sep 17, 2026

Copy link
Copy Markdown
Member

Replaces the shell-based wheel build system with a Python one built around
build_wheels.py, a TargetPlatform value object, and a platform-builder
hierarchy, and adopts pixi and pre-commit. Supersedes #302, which was built
from the same work before testing on three platforms uncovered sixteen
defects.

Two changes narrow supported platforms. Both are deliberate and need a
maintainer decision, not just a code review:

  • Linux wheels now carry exactly one platform tag. auditwheel repair
    runs with --only-plat --plat manylinux_2_28_<arch> on both
    architectures. Wheels stop installing on glibc 2.27 systems
    (Ubuntu 18.04, EOL April 2023).
  • The macOS floor moves from 11.0 to 14.0. No Apple Silicon hardware is
    orphaned — every M1 and later runs Sonoma — but macOS 11 through 13 users
    lose binary wheels.

Windows is unbuilt. It is the one platform with no test coverage here.
Two Windows defects are fixed in this branch but never exercised, so a
Windows run is needed before release.

Verification — what was actually built and checked

Every platform below was rebuilt from the committed tip of this branch
(d2ec481), after all sixteen fixes, producing seven cp311-abi3 wheels
each. The remote builder asserts the expected commit before building, so
these are not results from a stale checkout.

Platform Wheel tag Cache
Linux x86_64 cp311-abi3-manylinux_2_28_x86_64 383 MB
Linux aarch64 cp311-abi3-manylinux_2_28_aarch64 370 MB
macOS arm64 cp311-abi3-macosx_14_0_arm64 244 MB
Windows x86_64 not built not published

Checked on each: auditwheel repair accounting (6 started, 6 written), all
seven build steps reaching done, and — on all three platforms — an import
test that first uninstalled the seven previously installed itk* packages
and then reported Successfully installed, rather than the
Requirement already satisfied that would mean it had validated stale
packages instead of the wheels just built.

88 unit tests. pre-commit run --all-files clean.

Why the wheel fixup prepends the build environment's lib directory

One line in the Linux fixup will look arbitrary without this.

LD_LIBRARY_PATH for the repair step puts the build environment's own lib
first. Without it, pixi-provided binaries resolve against the container's
system libraries and patchelf fails to start with
GLIBCXX_3.4.29' not found: the manylinux_2_28 image supplies at most
3.4.25, the build environment 3.4.36. auditwheel then cannot construct
its ELF patcher and every repair fails.

The ordering is the point, and the obvious alternative does not work:
dropping /usr/lib64 instead was tested and still fails, because the binary
has no effective RUNPATH back to the environment and falls through to
/lib64.

aarch64 is insensitive to the ordering because its patchelf predates the
symbols in question — so an x86_64-only failure here does not imply an
x86_64-only cause.

Why macOS 14.0, and why it is set in three places

ITK 5.4.x deliberately set 11.0: itk-core 5.4.7 was built on SDK 15.2 and
carries minos 11.0. The published ITK 6 betas ship macosx_15_0_arm64
with minos tracking the build SDK exactly, which is the signature of no
deployment target being set at all — so upstream's 15.0 looks inherited from
its runner rather than chosen, and excludes macOS 14 users for no technical
reason.

14.0 is the lowest floor with no end-of-life macOS release below it, and it
orphans no Apple Silicon hardware.

Setting MACOSX_DEPLOYMENT_TARGET alone is not sufficient.
sysconfig.get_platform() reports the interpreter's build target and
ignores that variable, so the compile would produce minos 14.0 binaries
tagged macosx_11_0 — installable on macOS 11 through 13 and failing in
dyld at import. _PYTHON_HOST_PLATFORM is derived from the same value and
is what actually moves the tag. A test binds the three copies of the version
together so they cannot drift.

Known limitations a reviewer should weigh
  • The error paths have unit-test evidence only. Six of the fixes here
    make previously silent failures raise. Those paths execute only when a
    command fails, so no green build exercises them. They are covered by tests
    that fail against the pre-fix tree, but proving them live needs deliberate
    failure injection — a corrupted wheel before repair, a bad ITK tag, a
    failing zstd.
  • Windows is unbuilt, as above.
  • git bisect run pytest needs rc=5 treated as success. The first
    commit adopts pixi and pre-commit and introduces no symbol any test
    exercises, so pytest reports "no tests ran" there. Every other commit
    collects and passes.
  • manylinux_2_34 remains reachable in the platform table as headroom,
    although the shipping decision is manylinux_2_28. It is commented as such
    rather than removed.
Commit structure and dependency pinning

Twelve commits, each independently testable, ordered so the branch bisects.

Remote-module wheels are pinned to itk >= 6.0, < 7. Note that an ITK
release candidate does not satisfy that constraint under default pip rules;
installing against an RC needs --pre.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant