Skip to content

BUG: f2py reparses tokenized argv entries containing spaces #31958

Description

@VectorPeak

Describe the issue:

F2PY receives already-tokenized command-line arguments from normal entry points such as f2py and python -m numpy.f2py, but one newer-option preprocessing path rebuilds that argv list as a string and then splits it on whitespace again.

In numpy/f2py/f2py2e.py, get_newer_options() currently does:

iline = (' '.join(iline)).split()

That can destroy valid argv tokens that contain spaces. This matters for realistic Windows paths such as user profile directories, OneDrive for Business sync roots such as OneDrive - <organization name>, Program Files, include directories, and build directories. The issue is not that users pass an unquoted shell string; the shell/Python layer can already preserve quoted paths as one sys.argv element, and F2PY then splits that element internally.

This is reachable from normal user entry points:

f2py / python -m numpy.f2py
  -> numpy.f2py.f2py2e:main()
  -> run_main(sys.argv[1:])        # non-compile/signature-generation path
  -> scaninputline(comline_list)
  -> get_newer_options(inputline)

The compile path also passes sources = sys.argv[1:] to get_newer_options(sources) later in run_compile(). There appear to be related argv-boundary risks in the Meson wrapper-generation path, where run_compile() constructs a command string with ' '.join(...) and then calls .split() before run_main().

Reproduce the code example:

from numpy.f2py.f2py2e import get_newer_options

cases = [
    ["-I", r"C:\Users\First Last\include", r"C:\src\fib.f90"],
    ["--build-dir", r"C:\Users\First Last\f2py build", "-m", "fib", r"C:\src\fib.f90"],
    ["--freethreading-compatible", r"C:\Users\First Last\src\fib.f90"],
]

for case in cases:
    print("IN =", case)
    print("OUT=", get_newer_options(case))

On Windows, an additional path-list example is:

from numpy.f2py.f2py2e import get_newer_options

print(get_newer_options([
    "--include-paths",
    r"C:\Users\First Last\include;D:\vendor headers",
    r"C:\src\fib.f90",
]))

Error message:

For the first example, the include directory token is split at the space:

IN = ['-I', 'C:\\Users\\First Last\\include', 'C:\\src\\fib.f90']
OUT= (['C:\\Users\\First'], None, ['Last\\include', 'C:\\src\\fib.f90'])

For a build directory, the remainder passed to the legacy parser is also split:

IN = ['--build-dir', 'C:\\Users\\First Last\\f2py build', '-m', 'fib', 'C:\\src\\fib.f90']
OUT= ([], None, ['--build-dir', 'C:\\Users\\First', 'Last\\f2py', 'build', '-m', 'fib', 'C:\\src\\fib.f90'])

Expected behavior: already-tokenized argv entries should remain intact when passed to argparse.parse_known_args(). For example, r"C:\Users\First Last\include" should stay one token, not become r"C:\Users\First" plus r"Last\include".

Python and NumPy Versions:

Observed while inspecting the current NumPy development checkout. The focused local reproduction used Python 3.12 in a conda environment and the repository source at numpy/f2py/f2py2e.py.

Runtime Environment:

This is a command-line argument preprocessing issue and does not require compiled NumPy artifacts to reproduce. The local reproduction used a Windows environment, but the underlying token-boundary problem is platform-independent whenever an argv element contains whitespace.

How does this issue affect you or how did you find it:

I found this while reviewing F2PY option parsing after looking at nearby path-handling behavior. The practical impact is that users who correctly quote a path containing spaces can still have that path split inside F2PY after Python has already preserved it as one sys.argv entry.

This is most realistic on Windows, where normal paths may contain spaces, for example under C:\Users\First Last\..., OneDrive - <organization name>\\..., or C:\\Program Files (x86)\\.... The narrow behavior I would expect to preserve is already-tokenized argv entries; this does not require changing how F2PY parses arbitrary shell command strings.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions