Skip to content

builtins: generate accurate __text_signature__ - #8512

Merged
youknowone merged 9 commits into
RustPython:mainfrom
leehanjeong:8383-text-signature-fix
Aug 18, 2026
Merged

builtins: generate accurate __text_signature__#8512
youknowone merged 9 commits into
RustPython:mainfrom
leehanjeong:8383-text-signature-fix

Conversation

@leehanjeong

@leehanjeong leehanjeong commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Problem

RustPython's #[pyfunction]s generate a __text_signature__ that isn't comparable to CPython's. Of the 45 builtin functions RustPython shares with CPython 3.14, none reported an identical inspect.signature() before this PR:

  • 43 carried a phantom module parameter that doesn't exist.
  • None marked their true positional-only parameters as such.
  • 2 (round, sum) emitted a signature string that isn't valid Python, so inspect.signature() raised ValueError: builtin has invalid signature. The same applies to os.pathconf, binascii.b2a_base64 and binascii.b2a_uu in the wider stdlib.

Tools that rely on inspect.signature() (unittest.mock.autospec, pydoc, IDE completion) act on this bad metadata.

Scope

This PR fixes everything reachable by changing crates/derive-impl/src/util.rs's signature generator alone, without touching FromArgs or adding new types. The #[derive(FromArgs)] struct case is deliberately left out, so #8383 stays open after this merges.

What changed

  • Drop the $module marker. CPython's C functions receive the module as __self__, so inspect strips a leading $module. RustPython's #[pyfunction]s take no module argument, so __self__ is always None and there was nothing to strip. The marker surfaced as a parameter that doesn't exist (len(module, /, obj) instead of len(obj)).
  • Mark parameters positional-only. Plain arguments bind through FuncArgs::take_positional, which never consults the keyword map: len(obj=[1, 2]) already raised TypeError, but inspect reported obj as POSITIONAL_OR_KEYWORD. Generated signatures now carry the / marker, except for *args/**kwargs and empty parameter lists, which cannot have one.
  • Give up cleanly on unnamed parameters. Some functions bind through a destructuring pattern (e.g. fn round(RoundArgs { number, ndigits }: RoundArgs, ..)). The generator used to stringify the Rust pattern verbatim, producing text that isn't valid Python. inspect.signature() still raises ValueError for these, but now with no signature found for builtin, matching how CPython reports a builtin it has no signature for, rather than builtin has invalid signature.
  • Rename 9 parameters to match CPython (bin, ord, divmod, setattr, delattr, hasattr, isinstance, issubclass, aiter). All positional-only, so the name is documentation only and renaming doesn't change behavior.
  • Supporting changes. test_module_level_callable_noargs in Lib/test/test_pydoc/test_pydoc.py now passes for real (the phantom module parameter was the cause), so its expectedFailure marker is dropped. The new snippet's "no signature" assertions are guarded to RustPython, since CPython has real Argument Clinic signatures for round/sum.

Before / after

Measured across the 45 builtin functions RustPython and CPython 3.14 share:

exact match ValueError phantom module
before 0 / 45 2 (invalid signature) 43
after 23 / 45 2 (no signature found) 0

The 22 that still differ fall into two groups that need different work.

CPython documents a signature we could match (13). Twelve of these hold their arguments in a type the signature generator can't see into: a #[derive(FromArgs)] struct (__import__, compile, eval, exec, open, pow, print, sorted, and round/sum, which now report no signature at all) or an OptionalArg whose default lives in the function body (format, input). The thirteenth, breakpoint, differs only because CPython names its keyword catch-all **kws and the generator hardcodes **kwargs for every FuncArgs function.

CPython has no signature at all, and RustPython reports one (9). __build_class__, anext, dir, getattr, iter, max, min, next and vars. Teaching FromArgs to report its parameters would not close this gap, since the generated signature is not what's wrong: matching CPython here means deciding to suppress a signature we are able to produce. test_autospec_on_bound_builtin_function stays expectedFailure for exactly this reason, via time.ctime.

Not in this PR

Two follow-ups would close most of the first group above:

  • FromArgs reporting its own parameters. Implementors would report the parameters they consume, mirroring how arity() already self-reports parameter count. The information is already there in each field's #[pyarg(...)] attribute; it just isn't reachable from the signature generator today.
  • OptionalArg's hidden defaults, which the same mechanism would have to carry.

The second group needs a separate decision about whether RustPython should suppress signatures CPython doesn't publish, which seems worth settling before either follow-up.

Two unrelated problems turned up while investigating, both out of scope here:

  • __doc__ carries the raw signature prefix. len.__doc__ is 'len(obj, /)\n--\n\nReturn the number of items in a container.'. get_doc_from_internal_doc (type.rs) strips this, but is only wired to PyType.__doc__, not to builtin_func.rs or descriptor.rs.
  • Methods have almost no signatures. pyclass.rs only attaches a generated signature when the method already has a doc comment, so list.append, str.split and dict.get all report __text_signature__ of None. This PR touches the #[pymethod] path only enough to keep it compiling.

Test plan

  • New extra_tests/snippets/builtin_signature.py, run under both CPython and RustPython.
  • cargo run --release -- -m test test_inspect test_pydoc test_unittest
  • cargo test -p rustpython-derive-impl
  • prek run --all-files

Developed with assistance from Claude Code (claude-opus-5)

Summary by CodeRabbit

  • Bug Fixes

    • Improved builtin function signatures for clearer, more accurate display.
    • Removed phantom module parameters from displayed signatures.
    • Added correct positional-only and variadic parameter formatting.
    • Omitted signatures when they cannot be generated reliably.
    • Preserved documentation accurately when signatures are unavailable or incomplete.
  • Tests

    • Added coverage for builtin signature formatting, parameter names, and edge cases.

CPython's C functions receive the module as their first argument, so
PyCFunction.__self__ is the module and inspect strips the $module
parameter when building a Signature. A #[pyfunction] takes no such
argument, PyNativeFunction::zelf is None, and inspect has nothing to
strip, so the marker surfaced as a parameter that does not exist:

    inspect.signature(len)
    (module, /, obj)     # was
    (obj)                # now

All 45 builtins shared with CPython carried it. Methods are unaffected;
their $self marker comes from func_sig and both branches now produce the
same string.

Assisted-by: Claude Code:claude-opus-5
Arguments bind through `FuncArgs::take_positional`, which pops from the
positional list and never consults the keyword map, so a #[pyfunction]
argument cannot be passed by name:

    >>> len(obj=[1, 2])
    TypeError

The generated signature omitted the `/` marker, so inspect reported those
parameters as POSITIONAL_OR_KEYWORD, contradicting the call above. Emit
the marker, except for `*args`/`**kwargs`, which cannot be followed by
`/`, and for empty parameter lists.

14 of the 45 builtins shared with CPython now report an identical
signature, up from 0.

Assisted-by: Claude Code:claude-opus-5
Arguments bound by a destructuring pattern, e.g.

    fn round(RoundArgs { number, ndigits }: RoundArgs, ..)

have no name to report, and func_sig stringified the pattern verbatim:

    >>> round.__text_signature__
    '($module, RoundArgs { number, ndigits })'

That is not valid Python, so inspect.signature() raised "builtin has
invalid signature". Return None instead, which leaves
__text_signature__ unset and makes inspect raise "no signature found",
the same as for a CPython builtin that has no signature.

Affects round, sum, os.pathconf, binascii.b2a_base64 and
binascii.b2a_uu. Their docstrings are unchanged; only the signature
prefix is dropped.

Assisted-by: Claude Code:claude-opus-5
These parameters are positional-only, so their names only ever appear in
__text_signature__ and cannot be used at a call site. Naming them after
CPython makes the generated signatures directly comparable:

    bin        x            -> number
    ord        string       -> character
    divmod     a, b         -> x, y
    setattr    attr         -> name
    delattr    attr         -> name
    hasattr    attr         -> name
    isinstance typ          -> class_or_tuple
    issubclass subclass,typ -> cls, class_or_tuple
    aiter      iter_target  -> async_iterable

23 of the 45 builtins shared with CPython now report an identical
signature, up from 0 before this branch. The remainder need FromArgs to
report the parameters of its own structs, which is left for a follow-up.

Add extra_tests/snippets/builtin_signature.py covering the phantom
module parameter, the positional-only marker, the names above, and the
signature-less builtins.

Assisted-by: Claude Code:claude-opus-5
pydoc's summary line for time.time was "time(module)" because the
generated signature carried a $module parameter that inspect could not
strip. It now reads "time()", as the test expects.

Assisted-by: Claude Code:claude-opus-5
test_snippets runs every snippet under CPython as well, and CPython does
have Argument Clinic signatures for round and sum, so that block only
holds for RustPython.

Assisted-by: Claude Code:claude-opus-5
@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: d4d5938c-456f-41b7-a1d8-d9c94ebe7a40

📥 Commits

Reviewing files that changed from the base of the PR and between ce81d4d and 611744d.

📒 Files selected for processing (1)
  • crates/vm/src/stdlib/builtins.rs
🚧 Files skipped from review as they are similar to previous changes (1)
  • crates/vm/src/stdlib/builtins.rs

Included review availability: Your plan includes up to 10 reviews per rolling hour; 9 remain after this review.


📝 Walkthrough

Walkthrough

Builtin signature generation now supports missing signatures and positional-only formatting. Method generation preserves available documentation and emits no documentation when none exists. Builtin parameter names and signature tests were updated.

Changes

Builtin signature and documentation handling

Layer / File(s) Summary
Optional builtin signature generation
crates/derive-impl/src/util.rs, crates/vm/src/stdlib/builtins.rs, extra_tests/snippets/builtin_signature.py
func_sig and text_signature now return optional signatures, omit $module, add positional-only syntax, and reject destructuring patterns. Builtin parameter names and signature assertions were updated.
Optional method documentation
crates/derive-impl/src/pyclass.rs, crates/derive-impl/src/pymodule.rs
Generated methods preserve signature-only or source-only documentation and emit None when no documentation exists.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 61174

This PR corrects builtin signature metadata and related tests without introducing a concrete merge-blocking correctness, security, availability, or deployment risk; no actionable merge-blocking risk remains after normal checks and review.

Sequence Diagram(s)

sequenceDiagram
  participant Builtins as Builtin definitions
  participant Generator as Signature generator
  participant Methods as Method generation
  participant Inspect as inspect.signature
  Builtins->>Generator: Provide Rust function signature
  Generator-->>Methods: Return text signature or None
  Methods->>Methods: Combine signature and source documentation
  Methods-->>Inspect: Expose optional documentation and signature metadata
  Inspect-->>Builtins: Bind positional-only and variadic parameters
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: improving generated builtin text_signature metadata.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[ ] lib: cpython/Lib/pydoc.py
[ ] lib: cpython/Lib/pydoc_data
[ ] test: cpython/Lib/test/test_pydoc (TODO: 31)

dependencies:

  • pydoc (native: _pyrepl.pager, builtins, email.message, http.server, importlib._bootstrap, importlib._bootstrap_external, importlib.machinery, importlib.util, pydoc_data.topics, select, sys, time, urllib.parse)
    • pydoc_data
    • collections (native: _collections, _weakref, itertools, sys)
    • inspect (native: builtins, collections.abc, importlib.machinery, itertools, sys)
    • io (native: _io, _thread, errno, msvcrt, sys)
    • platform (native: _wmi, itertools, java.lang, sys, vms_lib, winreg)
    • pydoc_data
    • sysconfig (native: _sysconfig, _winapi, importlib.machinery, importlib.util, os.path, sys)
    • warnings (native: _contextvars, _thread, _warnings, builtins, sys)
    • future, annotationlib, ast, getopt, os, pkgutil, re, reprlib, textwrap, threading, tokenize, traceback, webbrowser

dependent tests: (5 tests)

  • pydoc: test_enum test_pydoc
    • pdb: test_pdb
    • xmlrpc.server: test_docxmlrpc test_xmlrpc

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@ShaharNaveh

Copy link
Copy Markdown
Contributor

@moreal the OSCCA job is failing

@youknowone youknowone added the z-ca-2026 Tag to track Contribution Academy 2026 label Aug 13, 2026

@youknowone youknowone left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great improvements, thanks!

Comment thread crates/derive-impl/src/util.rs Outdated
@youknowone
youknowone enabled auto-merge (squash) August 18, 2026 04:05
youknowone and others added 2 commits August 18, 2026 13:05
The merge of main took ord's signature from this branch, which renamed
the parameter to character, and its body from main, which rewrote ord to
accept bytes and bytearray through a parameter named c. The body then
referenced a name that no longer existed and the build failed.

Assisted-by: Claude Code:claude-opus-5
auto-merge was automatically disabled August 18, 2026 06:09

Head branch was pushed to by a user without write access

@youknowone
youknowone merged commit d8bb7bb into RustPython:main Aug 18, 2026
28 checks passed
JamesClarke7283 added a commit to JamesClarke7283/RustPython that referenced this pull request Aug 19, 2026
#[pyfunction]/#[pymethod] derive __text_signature__ from the Rust
parameter list, and a function that takes FuncArgs to check its own
arity has no parameters to report, so func_sig emits
"(*args, **kwargs)". Every builtin this branch rewrote that way -
len, abs, hash, chr, callable, bin, ord, divmod, isinstance,
issubclass and the rest of the 27 - stopped reporting the signature
that RustPython#8512 had just made accurate:

    inspect.signature(len)
    (*args, **kwargs)     # was (obj, /)

Add `text_signature = "..."`, which overrides the derived parameter
list, and give the affected builtins CPython's own, verified against
CPython 3.14.7. round declares (number, ndigits=None) and so has a
signature now, where before its destructuring pattern left it with
none; builtin_signature.py keeps sum as the signature-less case and
asserts round's instead.

The derived signature is still used wherever no override is given, so
genuinely variadic builtins such as breakpoint keep reporting
(*args, **kwargs).

Assisted-by: Claude:Claude Opus 5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

z-ca-2026 Tag to track Contribution Academy 2026

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants