Bug report
Bug description:
In the free-threaded build, BytesIO.__setstate__() installs the instance __dict__ with a plain (non-atomic) store, which races the lock-free LOAD_ATTR method-cache check that reads the same instance-dict slot with an atomic acquire load.
bytesio_setstate_lock_held() sets self->dict directly,
|
dict = PyTuple_GET_ITEM(state, 2); |
|
if (dict != Py_None) { |
|
if (!PyDict_Check(dict)) { |
|
PyErr_Format(PyExc_TypeError, |
|
"third item of state should be a dict, got a %.200s", |
|
Py_TYPE(dict)->tp_name); |
|
return NULL; |
|
} |
|
if (self->dict) { |
|
/* Alternatively, we could replace the internal dictionary |
|
completely. However, it seems more practical to just update it. */ |
|
if (PyDict_Update(self->dict, dict) < 0) |
|
return NULL; |
|
} |
|
else { |
|
self->dict = Py_NewRef(dict); |
|
} |
|
} |
but calling a method on the object (e.g. bio.read(...)) goes through a LOAD_ATTR method specialization that reads the instance-dict slot lock-free with an atomic acquire load.
|
// _CHECK_ATTR_METHOD_LAZY_DICT |
|
{ |
|
uint16_t dictoffset = read_u16(&this_instr[4].cache); |
|
char *ptr = ((char *)PyStackRef_AsPyObjectBorrow(owner)) + MANAGED_DICT_OFFSET + dictoffset; |
|
PyObject *dict = FT_ATOMIC_LOAD_PTR_ACQUIRE(*(PyObject **)ptr); |
|
if (dict != NULL) { |
|
UPDATE_MISS_STATS(LOAD_ATTR); |
|
assert(_PyOpcode_Deopt[opcode] == (LOAD_ATTR)); |
__setstate__ holds the BytesIO critical section, but the LOAD_ATTR fast path does not take the object lock, so the plain store races the atomic load on the dict slot.
Reproducer:
import io
from threading import Thread, Barrier
shared = io.BytesIO(b'initial payload')
states = [(b'A' * 64, 0, {}), (b'B' * 128, 32, {}), (b'C' * 256, 128, {})]
N_W, N_R = 1, 8
barrier = Barrier(N_W + N_R)
def setter():
barrier.wait()
for i in range(20000):
try:
shared.__setstate__(states[i % len(states)])
except Exception:
pass
def reader():
barrier.wait()
for i in range(20000):
try:
shared.read(8)
except Exception:
pass
if __name__ == "__main__":
threads = [Thread(target=setter) for _ in range(N_W)]
threads += [Thread(target=reader) for _ in range(N_R)]
for t in threads: t.start()
for t in threads: t.join()TSAN Report:
==================
WARNING: ThreadSanitizer: data race (pid=37676)
Write of size 8 at 0x7fffb66706c8 by thread T2:
#0 bytesio_setstate_lock_held /cpython/./Modules/_io/bytesio.c:968:24
#1 bytesio_setstate /cpython/./Modules/_io/bytesio.c:980:11
#2 method_vectorcall_O /cpython/Objects/descrobject.c:476:24
#3 _PyObject_VectorcallTstate /cpython/./Include/internal/pycore_call.h:144:11
#4 PyObject_Vectorcall /cpython/Objects/call.c:327:12
#5 _Py_VectorCallInstrumentation_StackRefSteal /cpython/Python/ceval.c:768:11
#6 _PyEval_EvalFrameDefault /cpython/Python/generated_cases.c.h:1906:35
...
Previous atomic read of size 8 at 0x7fffb66706c8 by thread T10:
#0 _Py_atomic_load_ptr_acquire /cpython/./Include/cpython/pyatomic_gcc.h:557:18
#1 _PyEval_EvalFrameDefault /cpython/Python/generated_cases.c.h:9149:34
#2 _PyEval_EvalFrame /cpython/./Include/internal/pycore_ceval.h:122:16
...
SUMMARY: ThreadSanitizer: data race /cpython/./Modules/_io/bytesio.c:968:24 in bytesio_setstate_lock_held
==================
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Linked PRs
Bug report
Bug description:
In the free-threaded build,
BytesIO.__setstate__()installs the instance__dict__with a plain (non-atomic) store, which races the lock-free LOAD_ATTR method-cache check that reads the same instance-dict slot with an atomic acquire load.bytesio_setstate_lock_held()setsself->dictdirectly,cpython/Modules/_io/bytesio.c
Lines 1042 to 1059 in a74280e
but calling a method on the object (e.g.
bio.read(...)) goes through a LOAD_ATTR method specialization that reads the instance-dict slot lock-free with an atomic acquire load.cpython/Python/generated_cases.c.h
Lines 9055 to 9062 in a74280e
__setstate__holds the BytesIO critical section, but the LOAD_ATTR fast path does not take the object lock, so the plain store races the atomic load on the dict slot.Reproducer:
TSAN Report:
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Linked PRs