Benchmark every pull request against its base branch - #1326
Open
nigrosimone wants to merge 1 commit into
Open
nigrosimone wants to merge 1 commit into
nigrosimone wants to merge 1 commit into
Conversation
nigrosimone
marked this pull request as ready for review
September 19, 2026 06:22
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A performance PR here comes with numbers measured by its author, on his machine, with his method, and the review can only take them or leave them. This adds a workflow that measures the PR itself, on the runner, against its base branch.
It builds both, starts a server from each build and alternates the load between them scenario by scenario, so both arms see the same machine in the same minute and only the ratio is read. Each arm runs twice: base against base and head against head is the noise of that run, and a row is marked only when it moved more than that. Every server reports its own cpu time too, so the table shows how busy it was and what a request cost it, the check the benchmarks README asks for before reading a number. wrk for the http rows,
load_testfrom the submodule for the ws rows, the server pinned to a core whose hyperthread stays idle.Scenarios: hello world, headers in and out, a JSON POST through
onData, a microcached route, ws echo at 20 B and 4 KB. Same design as brianc/node-postgres#3775.The table is posted as a comment on the PR, in the job summary when the PR comes from a fork like this one. About 10 minutes per run. Same code on both arms on a hosted runner (nigrosimone#1):
Node v26.9.0, AMD EPYC 7763, 4 cores (server on cpu 0, load on 2,3), wrk -t2 -c100.
#1325 against master stays inside the band on every row: nigrosimone#2.