Conversation
Bazel reports source symlinks as regular files, causing Python executable aliases to be copied into zipapps as full binaries. Hermetic runtimes also include shared libpython artifacts even when the interpreter links Python statically. Preserve source symlinks through sandbox indirection and exclude libpython shared objects from runtime files while retaining them for explicit native dependencies. Add regression coverage for archive structure and native extension loading.
There was a problem hiding this comment.
🟡 Changes recommended
The new archive assertion also runs on macOS, where the runtime intentionally retains libpython dylibs.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
This PR reduces zipapp sizes by preserving Python executable symlinks and excluding unused Linux libpython shared objects.
Changes:
- Detects source symlinks hidden behind Bazel sandbox indirection.
- Excludes shared
libpythonfiles from hermetic runtime files. - Adds archive and native-extension regression coverage.
File summaries
| File | Description |
|---|---|
tools/zipapp/zipper.py |
Preserves sandboxed source symlinks. |
tests/tools/zipapp/zipper_test.py |
Tests symlink archive behavior. |
tests/py_zipapp/venv_zipapp_test.py |
Checks archive structure and runtime contents. |
tests/py_zipapp/main.py |
Exercises native _ssl loading. |
python/private/hermetic_runtime_repo_setup.bzl |
Excludes Linux shared libpython objects. |
news/py-zipapp-size.fixed.md |
Documents the fix. |
Review details
- Files reviewed: 6/6 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Address PR review finding: the runtime-size assertion should mirror the Linux-only .so exclusion. macOS hermetic runtimes intentionally retain libpython dylibs, so only reject libpython shared-object entries. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
The two failures in buildkite seems to relate to using python 3.13.1. The transition where Astral switched python-build-standalone to statically link libpython directly into the bin/python executable occurred in May 2025. The custom A few options:
Thoughts? |
Custom hermetic distributions may use dynamically linked interpreters and require libpython.so.1.0 at runtime. The zipapp size optimization incorrectly removed that library from every Linux runtime. Restore the existing versioned-library inclusion and keep the zipapp optimization limited to preserving executable symlinks.
Astral Python Standalone builds from 20250604 onward include libpython statically in the interpreter, so packaging their shared libpython objects adds substantial unused size to self-contained zipapps. Recognize only official Astral release URLs at or after that build date and exclude libpython shared objects for those runtimes. Unknown, custom, and older distributions retain the existing versioned libraries for dynamically linked interpreters.
|
Implemented the Policy:
|
aignas
left a comment
There was a problem hiding this comment.
LGTM, but it would be good to expose a python extension configuration to disable the auto-detection. Please let me know if you would like to do this as part of this PR.
I am OK to just do python_repository attribute with an "auto" default where the user can very easily do a patch for now to override this if needed.
| # buildifier: disable=function-docstring-args | ||
| # buildifier: disable=function-docstring-return |
There was a problem hiding this comment.
This is saving a little bit of time instead of readability for later. Consider adding a comment that includes type-hints for the args, since this would be a trivial addition.
| return any([ | ||
| url.startswith(prefix) | ||
| for url in urls | ||
| for prefix in _ASTRAL_RELEASE_URL_PREFIXES |
There was a problem hiding this comment.
Is the thinking here that only astral releases are supported and if someone is re-hosting this from their local, they will not be able to benefit from this?
I am thinking that it probably makes sense to expose a flag in the python extension to set the static library detection to a particular value. The extension could have static_libpython_build="auto", "yes", "no" parameters where we use this function if it is "auto". Do you think it is complex to set it?
| *python_version_info | ||
| ) | ||
| urls = rctx.attr.urls or [rctx.attr.url] | ||
| interpreter_has_static_libpython = is_astral_static_libpython_build(urls, release_filename) |
There was a problem hiding this comment.
| interpreter_has_static_libpython = is_astral_static_libpython_build(urls, release_filename) | |
| if rctx.attr.static_libpython_build == "auto": | |
| interpreter_has_static_libpython = is_astral_static_libpython_build(urls, release_filename) | |
| elif rctx.attr.static_libpython_build == "yes": | |
| interpreter_has_static_libpython = True | |
| else: | |
| interpreter_has_static_libpython = False |
The static libpython transition started with the 20250517 Astral build, not 20250604. Also allow users to force inclusion or exclusion when auto-detection does not match a mirrored or customized runtime. Expose the auto/include/exclude mode through the Python extension overrides and document the Starlark helper argument types.
Document the auto, include, and exclude modes exposed by the Python extension, including the 20250517 Astral cutoff and fallback behavior for unknown runtimes. Also standardize indentation in the generated hermetic runtime BUILD definition.
Bazel reports source symlinks as regular files, causing Python executable aliases to be copied into zipapps as full binaries. Hermetic runtimes also include shared libpython artifacts even when the interpreter links Python statically.
Preserve source symlinks through sandbox indirection and exclude libpython shared objects from runtime files while retaining them for explicit native dependencies. Add regression coverage for archive structure and native extension loading.
Fixes #3534
Fixes #4163