Skip to content

Fix: Connection pool should not close connections assigned to queued requests - #1114

Closed
nishant1000 wants to merge 1 commit into
encode:masterfrom
nishant1000:fix/connection-pool-close-assigned-connection
Closed

nishant1000 wants to merge 1 commit into
encode:masterfrom
nishant1000:fix/connection-pool-close-assigned-connection

Conversation

@nishant1000

Copy link
Copy Markdown

Summary

Fixes #1110

Problem

There is a race condition in the connection pool where _assign_requests_to_connections() can close a connection that has already been assigned to a queued request.

The race condition:

  1. Thread A assigns an idle connection to a queued PoolRequest via �ssign_to_connection(). The connection remains IDLE because handle_request() hasn't been called yet.
  2. Thread B calls _assign_requests_to_connections() and sees the connection as still IDLE. It closes the connection as surplus keep-alive or to make room for a new connection.
  3. Thread A wakes up and calls connection.handle_request() on the now-closed connection:
    • If the request hasn't started → raises ConnectionNotAvailable → request re-queued
    • If the request has started → the socket is closed under a request waiting for its response → hangs until read timeout

Fix

In the surplus idle connection cleanup loop within _assign_requests_to_connections(), skip connections that have been assigned to any request in the pool.

Added _is_connection_assigned() helper method to both ConnectionPool (sync) and AsyncConnectionPool (async) that checks whether a connection is referenced by any request in the pool's request list.

Changes

  • httpcore/_sync/connection_pool.py: Added _is_connection_assigned() method; modified surplus idle cleanup to exclude assigned connections
  • httpcore/_async/connection_pool.py: Same fix for async variant
  • ests/_sync/test_connection_pool_issue1110.py: Regression tests (4 tests)
  • ests/_async/test_connection_pool_issue1110.py: Async regression tests (4 tests)

Testing

All 12 new tests pass. All existing sync connection pool tests pass (4 failures are pre-existing: 3 missing h2 module, 1 keepalive_expiry timing).

…requests (encode#1110)

The race condition occurs when:
1. Thread A assigns an idle connection to a queued PoolRequest
2. Thread B's _assign_requests_to_connections() sees it as idle and closes it
3. Thread A's handle_request() fails on the now-closed connection

Fix: In the surplus idle connection cleanup loop, skip connections that have
been assigned to any request in the pool. Added _is_connection_assigned()
helper method to both sync and async connection pools.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Connection pool can close a connection it has already assigned to a queued request

1 participant