Skip to content

fix(cloudflare): Enforce flush timeout across Workflow lifecycle - #24483

Open
matthewbjones wants to merge 3 commits into
getsentry:developfrom
matthewbjones:feat/cloudflare-enforce-flush-timeout
Open

matthewbjones wants to merge 3 commits into
getsentry:developfrom
matthewbjones:feat/cloudflare-enforce-flush-timeout

Conversation

@matthewbjones

Copy link
Copy Markdown

Cloudflare was reporting that Sentry-owned waitUntil() tasks remained pending after instrumented Workflows had completed. The configured flush(2000) timeout did not fully bound the work associated with a flush.

Three separate lifecycle gaps contributed to the warning.

Transport requests survived a timed-out drain

IsolatedPromiseBuffer.drain(timeout) returned false when its timer expired, but the request producers it had started continued running. A slow or stalled fetch therefore remained attached to the invocation after Sentry reported that its flush had timed out.

Each drain now owns an AbortController. When that drain expires, it aborts only the requests started by that drain. makeCloudflareTransport combines the drain signal with any caller-provided fetchOptions.signal, removes both listeners when the request settles, and keeps overlapping drains isolated so one timeout cannot abort another drain's requests.

Workflow steps inherited the previous step's flush point

Cached Cloudflare clients reuse the Workflow run's isolation scope across step RPC invocations. The first boundary flush marked that scope's flushPointReached state as true, and later steps inherited it. Envelopes created by those later steps were consequently treated as post-invocation telemetry and registered additional eager waitUntil(transport.flush(2000)) drains even though every step already performs its own boundary flush.

The wrapped step callback now resets flushPointReached before it captures telemetry. This makes each Workflow step begin before its own flush point, while retaining eager delivery for telemetry that is genuinely created after that step's boundary flush.

Flush-lock finalization was outside the timeout

CloudflareClient.flush(timeout) awaited flushLock.finalize() before applying timeout to pending spans and the transport. A user waitUntil task that never settled could therefore keep the Sentry flush pending indefinitely. The pending-span and transport phases could also each receive the original timeout instead of the time remaining from one overall deadline.

flush() now computes one deadline and passes the remaining budget through flush-lock finalization, pending-span completion, and the transport drain. It returns false without beginning a later phase when the budget has already been exhausted.

Validation

Each failure has focused red/green regression coverage:

  • a timed-out drain aborts its request, preserves a caller signal, and does not abort requests owned by an overlapping drain;
  • every Workflow step starts with its own unreached flush point; and
  • a never-settling flush lock causes flush(timeout) to return false within the deadline.

The production reproduction uses 100 trivial step.do() calls and an HTTP transport that delays every response for 120 seconds. The transport fix alone removed the warnings with cacheClient: false, but warnings remained on the current cached-client path. Disabling eager envelope delivery isolated the remaining behavior to the inherited Workflow flush point. With eager delivery enabled and both lifecycle fixes applied, 20/20 deployed Workflow instances completed with zero matching waitUntil() warnings in Cloudflare Observability.

The complete @sentry/cloudflare suite passes (955/955), along with package lint, build, and typecheck.

  • If you've added code that should be tested, please add tests.
  • Ensure your code lints and the test suite passes (yarn lint) & (yarn test).
  • Link an issue if there is one related to your pull request. If no issue is linked, one will be auto-generated and linked.

Closes #24482

Cloudflare's flush deadline could expire while transport fetches remained active, Workflow steps could inherit the previous step's flush point and schedule redundant eager drains, and flush-lock finalization could wait outside the configured timeout.

Abort requests when a transport drain expires, reset the flush point at each Workflow step boundary, and share one deadline across flush-lock, pending-span, and transport phases.

Add focused regression coverage for all three paths and update timer-based tests to await the user task before asserting teardown.

Fixes getsentry#24482

Co-authored-by: OpenAI Codex <codex@openai.com>
@matthewbjones
matthewbjones force-pushed the feat/cloudflare-enforce-flush-timeout branch from 7ad0606 to 643d751 Compare September 17, 2026 22:04
@matthewbjones
matthewbjones marked this pull request as ready for review September 17, 2026 22:13
@matthewbjones
matthewbjones requested a review from a team as a code owner September 17, 2026 22:13
@matthewbjones
matthewbjones requested review from isaacs and mydea and removed request for a team September 17, 2026 22:13

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread packages/cloudflare/src/client.ts
Share one deadline across the flush lock, pending spans, client processing, and transport drain so pre-drain timeouts cannot strand buffered envelopes.

Add regression coverage for lock and span starvation and for the complete flush deadline.

Co-authored-by: OpenAI Codex <codex@openai.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit acd76c4. Configure here.

Comment thread packages/cloudflare/test/client.test.ts
Comment thread packages/cloudflare/test/client.test.ts
Use the Cloudflare test suite's established fake-timer cleanup and explicit timer advancement patterns for flush deadline assertions.

Co-authored-by: OpenAI Codex <codex@openai.com>
@msonnb
msonnb requested a review from JPeer264 September 18, 2026 07:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cloudflare Workflow instrumentation leaves pending waitUntil tasks after the flush timeout

1 participant