Describe the issue:
F2PY receives already-tokenized command-line arguments from normal entry points such as f2py and python -m numpy.f2py, but one newer-option preprocessing path rebuilds that argv list as a string and then splits it on whitespace again.
In numpy/f2py/f2py2e.py, get_newer_options() currently does:
iline = (' '.join(iline)).split()
That can destroy valid argv tokens that contain spaces. This matters for realistic Windows paths such as user profile directories, OneDrive for Business sync roots such as OneDrive - <organization name>, Program Files, include directories, and build directories. The issue is not that users pass an unquoted shell string; the shell/Python layer can already preserve quoted paths as one sys.argv element, and F2PY then splits that element internally.
This is reachable from normal user entry points:
f2py / python -m numpy.f2py
-> numpy.f2py.f2py2e:main()
-> run_main(sys.argv[1:]) # non-compile/signature-generation path
-> scaninputline(comline_list)
-> get_newer_options(inputline)
The compile path also passes sources = sys.argv[1:] to get_newer_options(sources) later in run_compile(). There appear to be related argv-boundary risks in the Meson wrapper-generation path, where run_compile() constructs a command string with ' '.join(...) and then calls .split() before run_main().
Reproduce the code example:
from numpy.f2py.f2py2e import get_newer_options
cases = [
["-I", r"C:\Users\First Last\include", r"C:\src\fib.f90"],
["--build-dir", r"C:\Users\First Last\f2py build", "-m", "fib", r"C:\src\fib.f90"],
["--freethreading-compatible", r"C:\Users\First Last\src\fib.f90"],
]
for case in cases:
print("IN =", case)
print("OUT=", get_newer_options(case))
On Windows, an additional path-list example is:
from numpy.f2py.f2py2e import get_newer_options
print(get_newer_options([
"--include-paths",
r"C:\Users\First Last\include;D:\vendor headers",
r"C:\src\fib.f90",
]))
Error message:
For the first example, the include directory token is split at the space:
IN = ['-I', 'C:\\Users\\First Last\\include', 'C:\\src\\fib.f90']
OUT= (['C:\\Users\\First'], None, ['Last\\include', 'C:\\src\\fib.f90'])
For a build directory, the remainder passed to the legacy parser is also split:
IN = ['--build-dir', 'C:\\Users\\First Last\\f2py build', '-m', 'fib', 'C:\\src\\fib.f90']
OUT= ([], None, ['--build-dir', 'C:\\Users\\First', 'Last\\f2py', 'build', '-m', 'fib', 'C:\\src\\fib.f90'])
Expected behavior: already-tokenized argv entries should remain intact when passed to argparse.parse_known_args(). For example, r"C:\Users\First Last\include" should stay one token, not become r"C:\Users\First" plus r"Last\include".
Python and NumPy Versions:
Observed while inspecting the current NumPy development checkout. The focused local reproduction used Python 3.12 in a conda environment and the repository source at numpy/f2py/f2py2e.py.
Runtime Environment:
This is a command-line argument preprocessing issue and does not require compiled NumPy artifacts to reproduce. The local reproduction used a Windows environment, but the underlying token-boundary problem is platform-independent whenever an argv element contains whitespace.
How does this issue affect you or how did you find it:
I found this while reviewing F2PY option parsing after looking at nearby path-handling behavior. The practical impact is that users who correctly quote a path containing spaces can still have that path split inside F2PY after Python has already preserved it as one sys.argv entry.
This is most realistic on Windows, where normal paths may contain spaces, for example under C:\Users\First Last\..., OneDrive - <organization name>\\..., or C:\\Program Files (x86)\\.... The narrow behavior I would expect to preserve is already-tokenized argv entries; this does not require changing how F2PY parses arbitrary shell command strings.
Describe the issue:
F2PY receives already-tokenized command-line arguments from normal entry points such as
f2pyandpython -m numpy.f2py, but one newer-option preprocessing path rebuilds that argv list as a string and then splits it on whitespace again.In
numpy/f2py/f2py2e.py,get_newer_options()currently does:That can destroy valid argv tokens that contain spaces. This matters for realistic Windows paths such as user profile directories, OneDrive for Business sync roots such as
OneDrive - <organization name>,Program Files, include directories, and build directories. The issue is not that users pass an unquoted shell string; the shell/Python layer can already preserve quoted paths as onesys.argvelement, and F2PY then splits that element internally.This is reachable from normal user entry points:
The compile path also passes
sources = sys.argv[1:]toget_newer_options(sources)later inrun_compile(). There appear to be related argv-boundary risks in the Meson wrapper-generation path, whererun_compile()constructs a command string with' '.join(...)and then calls.split()beforerun_main().Reproduce the code example:
On Windows, an additional path-list example is:
Error message:
For the first example, the include directory token is split at the space:
For a build directory, the remainder passed to the legacy parser is also split:
Expected behavior: already-tokenized argv entries should remain intact when passed to
argparse.parse_known_args(). For example,r"C:\Users\First Last\include"should stay one token, not becomer"C:\Users\First"plusr"Last\include".Python and NumPy Versions:
Observed while inspecting the current NumPy development checkout. The focused local reproduction used Python 3.12 in a conda environment and the repository source at
numpy/f2py/f2py2e.py.Runtime Environment:
This is a command-line argument preprocessing issue and does not require compiled NumPy artifacts to reproduce. The local reproduction used a Windows environment, but the underlying token-boundary problem is platform-independent whenever an argv element contains whitespace.
How does this issue affect you or how did you find it:
I found this while reviewing F2PY option parsing after looking at nearby path-handling behavior. The practical impact is that users who correctly quote a path containing spaces can still have that path split inside F2PY after Python has already preserved it as one
sys.argventry.This is most realistic on Windows, where normal paths may contain spaces, for example under
C:\Users\First Last\...,OneDrive - <organization name>\\..., orC:\\Program Files (x86)\\.... The narrow behavior I would expect to preserve is already-tokenized argv entries; this does not require changing how F2PY parses arbitrary shell command strings.