How do you use Sentry?
Sentry Saas (sentry.io)
Version
2.51.0
Steps to Reproduce
After upgrading to Python 3.14, our Celery tasks started failing with 'too many open files' errors. We found that
- Sentry's Celery integration leaks a file descriptor for every task executed
- Python 3.14's new GC is less aggressive and leaked file descriptors can hang around for much longer
The relevant code is:
|
with capture_internal_exceptions(): |
|
span.set_data( |
|
SPANDATA.MESSAGING_SYSTEM, |
|
task.app.connection().transport.driver_type, |
|
) |
Accessing the transport property will create a new redis connection. (We use a redis broker but I assume the behavior is the same for other transport types.) This connection is never closed so it's left to the GC to clean up.
To see this in action, take the "add" example from https://docs.sentry.io/platforms/python/integrations/celery/, configure a redis broker and change the main function to queue many tasks:
def main():
for _ in range(1000):
add.delay(4, 4)
Use Python 3.14, celery[redis]==5.6.2, sentry-sdk==2.51.0. Then check /proc/(worker pid)/fd/.
Expected Result
The Celery worker has few open files.
Actual Result
The Celery worker has hundreds of anon_inode:[eventpoll] files.
How do you use Sentry?
Sentry Saas (sentry.io)
Version
2.51.0
Steps to Reproduce
After upgrading to Python 3.14, our Celery tasks started failing with 'too many open files' errors. We found that
The relevant code is:
sentry-python/sentry_sdk/integrations/celery/__init__.py
Lines 394 to 398 in bf0a683
Accessing the
transportproperty will create a new redis connection. (We use a redis broker but I assume the behavior is the same for other transport types.) This connection is never closed so it's left to the GC to clean up.To see this in action, take the "add" example from https://docs.sentry.io/platforms/python/integrations/celery/, configure a redis broker and change the main function to queue many tasks:
Use Python 3.14,
celery[redis]==5.6.2,sentry-sdk==2.51.0. Then check/proc/(worker pid)/fd/.Expected Result
The Celery worker has few open files.
Actual Result
The Celery worker has hundreds of
anon_inode:[eventpoll]files.