Skip to content

fix: publish tutorials dependencies as PEP 621 extras - #885

Merged
SemyonSinchenko merged 1 commit into
graphframes:mainfrom
james-willis:fix/884-tutorials-extras
Aug 22, 2026
Merged

SemyonSinchenko merged 1 commit into
graphframes:mainfrom
james-willis:fix/884-tutorials-extras

Conversation

@james-willis

Copy link
Copy Markdown
Collaborator

What changes were proposed in this pull request?

  • python/pyproject.toml: declare click, py7zr and requests in [project.optional-dependencies] as the tutorials extra (plus an all alias), with the same version constraints they already had. The existing [tool.poetry.group.tutorials.dependencies] group is left in place so poetry install --with tutorials keeps working for contributors and the docs in 01-contributing-guide.md stay accurate.
  • python/poetry.lock: regenerated (Poetry 2.4.1) — marker/group metadata only, no package version changes.
  • python/graphframes/console.py: catch the ImportError in main() and log a message that names the extra, instead of letting the console script die with a bare traceback.

all repeats the requirement list rather than referencing graphframes-py[tutorials] because Poetry refuses to lock a self-referential extra (Package 'graphframes-py[tutorials]' is listed as a dependency of itself).

Why are the changes needed?

Fixes #884.

Poetry dependency groups are a local dev-environment feature — Poetry never writes them into built distribution metadata. Only [project.optional-dependencies] (PEP 621) or [tool.poetry.extras] emit Provides-Extra / Requires-Dist … ; extra == "…". So the published graphframes_py-0.12.1-py3-none-any.whl has zero Requires-Dist and zero Provides-Extra lines, while still installing an unconditional console script:

[console_scripts]
graphframes=graphframes.console:main

console.py imports click at module scope and graphframes/tutorials/download.py imports py7zr and requests, so the shipped graphframes binary is broken for everyone, and there is no extra to install that fixes it.

Before, on a clean venv:

$ pip install 'graphframes-py[tutorials]'
WARNING: graphframes-py 0.12.1 does not provide the extra 'tutorials'
$ graphframes
Traceback (most recent call last):
  ...
  File ".../graphframes/console.py", line 1, in <module>
    import click
ModuleNotFoundError: No module named 'click'

After, building the wheel from this branch:

$ unzip -p dist/graphframes_py-*.whl '*/METADATA' | grep -E '^(Requires-Dist|Provides-Extra)'
Provides-Extra: all
Provides-Extra: tutorials
Requires-Dist: click (>=8.1.8,<9.0.0) ; extra == "all"
Requires-Dist: click (>=8.1.8,<9.0.0) ; extra == "tutorials"
Requires-Dist: py7zr (>=0.22.0,<0.23.0) ; extra == "all"
Requires-Dist: py7zr (>=0.22.0,<0.23.0) ; extra == "tutorials"
Requires-Dist: requests (>=2.33.0,<3.0.0) ; extra == "all"
Requires-Dist: requests (>=2.33.0,<3.0.0) ; extra == "tutorials"
$ pip install 'graphframes_py-*.whl[tutorials]'
$ graphframes
Usage: graphframes [OPTIONS] COMMAND [ARGS]...

  GraphFrames CLI: a collection of commands for graphframes.

Options:
  --help  Show this message and exit.

Commands:
  stackexchange  Download Stack Exchange archive for a given SUBDOMAIN.

And without the extra, the failure is now actionable rather than a traceback:

$ pip install 'graphframes_py-*.whl'
$ graphframes
The `graphframes` CLI requires the optional `tutorials` dependencies, which are not installed. Install them with: pip install 'graphframes-py[tutorials]' (No module named 'click')
$ echo $?
1

Note on #881

#881 landed the lazy import in console.py but its pyproject.toml half was reverted, on the grounds that click "belongs under tutorials, since it is required for the CLI" and shouldn't become a separate one-package group. This PR keeps that structure — all three packages stay together under tutorials — and only changes where they are declared so they actually reach the published metadata. No new runtime dependency is added to the base install.

pyspark remains undeclared, unchanged by this PR.

`click`, `py7zr` and `requests` were declared in
`[tool.poetry.group.tutorials.dependencies]`. Poetry dependency *groups*
are a local dev-environment feature and are never written into built
distribution metadata, so the published wheel carried no `Provides-Extra`
and no `Requires-Dist` at all:

    $ pip install 'graphframes-py[tutorials]'
    WARNING: graphframes-py 0.12.1 does not provide the extra 'tutorials'
    $ graphframes
    ModuleNotFoundError: No module named 'click'

Declare the same three packages in `[project.optional-dependencies]` so
they land in the wheel metadata and `pip install graphframes-py[tutorials]`
(or `[all]`) works. The `tutorials` Poetry group is kept so that
`poetry install --with tutorials` still works for contributors.

Also catch the `ImportError` in `graphframes.console.main()` and log a
message pointing at the extra, instead of raising a bare traceback from
the console script that is installed for every user.

Fixes graphframes#884
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.63%. Comparing base (28d181c) to head (1910ae3).
⚠️ Report is 3 commits behind head on main.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #885      +/-   ##
==========================================
+ Coverage   79.26%   81.63%   +2.36%     
==========================================
  Files          81       81              
  Lines        4712     4682      -30     
  Branches      554      574      +20     
==========================================
+ Hits         3735     3822      +87     
+ Misses        977      860     -117     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@SemyonSinchenko
SemyonSinchenko merged commit eb6f25e into graphframes:main Aug 22, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: tutorials broken, graphframes CLI missing click, py7zr and requests as dependencies

3 participants