Skip to content

Prototype exc.__tracebacks__ - #2

Open
pelson wants to merge 4 commits into
mainfrom
feature/exc.__tracebacks__
Open

pelson wants to merge 4 commits into
mainfrom
feature/exc.__tracebacks__

Conversation

@pelson

@pelson pelson commented Jun 6, 2026

Copy link
Copy Markdown
Owner

Prototype for having multiple tracebacks in a chain on a single exception. This is not the same as exception groups nor exception causes.

pelson added 4 commits June 4, 2026 15:38
When `raise expr` is used to re-raise a previously captured exception
from an unrelated call chain (e.g. `Future.result()` re-raising an
exception stored on the future), Python prepends the current frame to
the exception's existing traceback. The rendered stack then blends two
unrelated call chains and looks like a single continuous call path that
never actually existed.

`do_raise` now drops the exception's existing traceback when its head
frame is not on the current call chain, so the rendered trace reflects
the actual raise site. A user-attached traceback (via
`__traceback__ = tb` or `with_traceback(tb)`) is preserved: the
Python-level setter sets a new `user_defined_traceback` flag on the
exception and the heuristic respects it. The flag fits into existing
alignment padding in `PyException_HEAD`, so struct sizes and field
offsets are unchanged.

The C API `PyException_SetTraceback` is unchanged in behaviour and
deliberately does not set the flag; it is the path the internal
auto-stitcher (`PyTraceBack_Here`) uses during exception propagation.
When a captured exception is re-raised from a foreign call chain, the
prior traceback is no longer dropped. It is appended to a new read-only
tuple attribute on BaseException, __traceback_history__ (oldest first),
and a fresh __traceback__ is built from the actual raise site. The
default exception renderer emits each history fragment, oldest first,
prefixed with the boundary line "During an earlier raise of this
exception, the traceback was:" before the familiar
"Traceback (most recent call last):" header on the current tb.

Builds on the foreign-tb heuristic and user_defined_traceback flag from
the prior commit: that commit dropped the foreign tb to stop blending
two unrelated stacks; this commit preserves the dropped fragment so the
information is still available to the user.

- Include/cpython/pyerrors.h: extend PyException_HEAD with
  traceback_history (PyObject*).
- Objects/exceptions.c: initialize in BaseException_new and
  BaseException_vectorcall; add to tp_traverse and tp_clear; add a
  read-only __traceback_history__ getter on BaseException_getset.
- Python/ceval.h (do_raise): split heuristic detection from the action;
  on foreign-tb, append e->traceback to e->traceback_history (immutable
  tuple replace) and then clear e->traceback so a fresh tb is built.
- Python/pythonrun.c: render history fragments before the current tb in
  print_exception_traceback.
- Lib/traceback.py: capture per-fragment StackSummary on
  TracebackException.traceback_history; emit boundary marker and
  fragments in TracebackException.format().
- Lib/test/test_exceptions.py: TracebackHistoryTests covers empty
  default, foreign re-raise appending, in-chain re-raise NOT appending,
  with_traceback NOT appending, accumulation across multiple foreign
  chains, read-only attribute, and history not leaking into
  __cause__/__context__.
Move the foreign-traceback heuristic from do_raise into _PyErr_Restore,
the lowest-level exception install path. Every raise -- the RAISE_VARARGS
bytecode (do_raise), PyErr_SetObject from C extensions like _asyncio,
gen.throw() / coroutine.throw(), and direct PyErr_Restore callers -- now
flows through the same logic without each path having to duplicate it.

This fixes a class of bugs that PyErr_SetObject-only placement missed.
In particular, asyncio's task wakeup path raises a stored exception by
calling coroutine.throw() which routes through gen_set_exception ->
PyErr_Restore, bypassing PyErr_SetObject entirely. With the heuristic
at the PyErr_Restore level, awaiting a task whose coroutine raised in a
foreign call chain now correctly moves the worker traceback onto
__traceback_history__.

- Python/errors.c: place the heuristic in _PyErr_Restore, immediately
  after the optional PyException_SetTraceback attachment and before
  _PyErr_SetRaisedException installs the exception.
- Python/ceval.h: drop the do_raise-local heuristic and its action
  block; do_raise now just sets cause and calls _PyErr_SetObject which
  forwards through _PyErr_Restore.
- Modules/_asynciomodule.c: simplified back to PyException_SetTraceback
  + PyErr_SetObject -- the central heuristic in _PyErr_Restore handles
  the foreign-tb case automatically.
- Lib/asyncio/futures.py: pure-Python Future.result() now uses bare
  `raise self._exception` instead of with_traceback, so the central
  heuristic gets to decide where the worker tb belongs.
- Lib/test/test_asyncio/test_futures2.py::test_future_traceback: update
  to reflect the new model -- the worker raise line now lives in
  __traceback_history__, and the current __traceback__ contains
  `await future`.
…__tracebacks__

Replace the read-only __traceback_history__ tuple introduced in the
previous commit with a unified __tracebacks__ property: an ordered tuple
of all traceback fragments contributed by this exception's raise history.
__traceback__ continues to alias the last fragment.

The user-facing model is "one exception, an ordered tuple of traceback
fragments, period". Fresh (never-raised) exceptions have
__tracebacks__ is None; after a raise the tuple grows by one entry per
foreign call chain crossing. The setter accepts None or a tuple of
TracebackType and rejects lists with TypeError -- the tuple is
intentionally immutable so captured references can't be mutated under
the holder. The supported user-side mutation is
`exc.__tracebacks__ += (tb,)`.

This is a *computed facade* over the existing sidecar fields
(self->traceback_history holds the prior fragments, self->traceback
holds the current one). The C struct layout is intentionally preserved
so that existing C extensions reading
((PyBaseExceptionObject *)exc)->traceback continue to work; only the
Python-visible attribute surface changes.

- Objects/exceptions.c: drop BaseException___traceback_history___get
  and its getset entry. Add BaseException___tracebacks___get (builds
  fresh tuple = traceback_history + (traceback,) on each read; returns
  None when traceback is NULL) and BaseException___tracebacks___set
  (validates tuple-of-TracebackType, splits last element into
  self->traceback and the rest into self->traceback_history; None or
  empty tuple clears both).
- Lib/traceback.py: TracebackException builds self._prior_stacks from
  exc.__tracebacks__[:-1] instead of capturing __traceback_history__;
  the renderer iterates _prior_stacks before the current stack.
- Lib/asyncio/futures.py: comment updated.
- Lib/test/test_exceptions.py: TracebackHistoryTests rewritten as
  TracebackTupleTests (13 cases: fresh-state None, single/foreign/
  multi-reraise fragment counts, in-chain reraise extends last,
  with_traceback preserves single fragment, __traceback__ setter
  replaces last fragment, __tracebacks__ setter accepts None/empty
  tuple, preserves order, rejects lists and non-TB elements, iadd
  sugar, no cause/context leakage).
- Lib/test/test_asyncio/test_futures2.py::test_future_traceback: walk
  __tracebacks__[:-1] instead of __traceback_history__.
- Lib/test/test_sys.py: BaseException sizeof string updated from '6Pb'
  to '7Pbb' (pre-existing baseline failure from the sidecar fields).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant