fix: detect space-separated AI model names in co-author trailers by stasimus · Pull Request #506 · commit-check/commit-check

stasimus · 2026-07-31T21:53:59Z

With ai_attribution = "forbid", trailers using space-separated model names pass undetected:

Co-Authored-By: Claude Opus 4.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Gemini 2.5 Pro <gemini@example.com>

The Claude pattern only matches Claude/Claude Code verbatim, and the generic pattern requires hyphenated ids (claude-sonnet-4). Claude Code emits such trailers in the wild (observed in our org).

Two new patterns:

any co-author with the noreply@anthropic.com email, regardless of display name;
space-separated model names with a purely numeric version token, so human names — including ordinals like Claude Dubois 3rd — stay unflagged.

Tests added for both, plus the ordinal false-positive case.

Summary by CodeRabbit

Bug Fixes
- Improved detection of AI-generated commit signatures attributed to Claude, including Anthropic email trailers and model-name variants.
- Added support for identifying space-separated model names with numeric versions, such as Claude Opus 4.5 and GPT 4 Turbo.
- Reduced false positives by distinguishing AI model signatures from human names containing ordinal suffixes.
Tests
- Added coverage for the expanded detection scenarios and human-name exclusions.

netlify · 2026-07-31T21:54:05Z

✅ Deploy Preview for commit-check ready!

Name	Link
🔨 Latest commit	`94c2a23`
🔍 Latest deploy log	https://app.netlify.com/projects/commit-check/deploys/6a6d1e03d569250008fb25a7
😎 Deploy Preview	https://deploy-preview-506--commit-check.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

coderabbitai · 2026-07-31T21:54:18Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 70b8a1cd-0ee5-4d1e-9967-80cd49cd8985

📥 Commits

Reviewing files that changed from the base of the PR and between 8fd6c1a and 94c2a23.

📒 Files selected for processing (2)

commit_check/ai_signatures_data.py
tests/ai_signatures_test.py

🚧 Files skipped from review as they are similar to previous changes (2)

commit_check/ai_signatures_data.py
tests/ai_signatures_test.py

📝 Walkthrough

Walkthrough

The change expands AI signature detection for Anthropic co-author trailers and space-separated Claude, GPT, and Gemini model names. Tests cover supported variants and reject a human-name false positive.

Changes

AI signature detection

Layer / File(s)	Summary
Claude Anthropic trailer detection `commit_check/ai_signatures_data.py`, `tests/ai_signatures_test.py`	Claude Code detection matches co-author names paired with `noreply@anthropic.com`. Tests cover Claude model-name variants.
Space-separated model detection `commit_check/ai_signatures_data.py`, `tests/ai_signatures_test.py`	A generic signature detects Claude, GPT, and Gemini model names with numeric versions, qualifiers, metadata, and email addresses. Tests reject ordinal human names.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

commit-check/commit-check#456: Extends the same AI signature patterns and test suite.

Suggested reviewers: shenxianpeng

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: detecting space-separated AI model names in co-author trailers.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

sonarqubecloud · 2026-07-31T21:54:39Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

coderabbitai

Actionable comments posted: 1

🧹 Nitpick comments (1)

tests/ai_signatures_test.py (1)
60-69: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Make the Claude case exercise the Generic AI pattern.

has_ai_signature only checks whether any pattern matches. The Claude fixture uses noreply@anthropic.com, so the Claude Code pattern makes this test pass even if the new space-separated Generic AI pattern is broken.

Use a non-Anthropic address for this case and assert that detect_ai_signatures returns tool == "Generic AI".
Proposed test adjustment
-            "Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>",
+            "Co-authored-by: Claude Sonnet 4.5 <claude@example.com>",
...
-            assert has_ai_signature(message), trailer
+            result = detect_ai_signatures(message)
+            assert any(s["tool"] == "Generic AI" for s in result), trailer
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/ai_signatures_test.py` around lines 60 - 69, Update
test_space_separated_model_name_detected to use a non-Anthropic address for the
Claude fixture, then call detect_ai_signatures and assert the matching result
has tool == "Generic AI"; keep the existing coverage for the other
space-separated model names.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@commit_check/ai_signatures_data.py`:
- Around line 240-241: Update the regex used by _trailer() so AI model
signatures cannot span lines: replace newline-capable whitespace with horizontal
whitespace and exclude newline characters from the model value, parenthesized
metadata, and angle-bracket email portions. Preserve existing matching for valid
single-line trailers such as GPT 4, optional annotations, and email values.

---

Nitpick comments:
In `@tests/ai_signatures_test.py`:
- Around line 60-69: Update test_space_separated_model_name_detected to use a
non-Anthropic address for the Claude fixture, then call detect_ai_signatures and
assert the matching result has tool == "Generic AI"; keep the existing coverage
for the other space-separated model names.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 5215b8df-8ba5-47d2-9c69-26df560d670f

📥 Commits

Reviewing files that changed from the base of the PR and between 91c4a9f and 8fd6c1a.

📒 Files selected for processing (2)

commit_check/ai_signatures_data.py
tests/ai_signatures_test.py

sonarqubecloud · 2026-07-31T22:13:49Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

codecov · 2026-07-31T22:15:29Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.36%. Comparing base (b75ec1c) to head (94c2a23).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #506   +/-   ##
=======================================
  Coverage   97.36%   97.36%           
=======================================
  Files          12       12           
  Lines        1175     1175           
=======================================
  Hits         1144     1144           
  Misses         31       31

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

shenxianpeng

Thanks for your PR!

codspeed-hq · 2026-07-31T22:27:24Z

Merging this PR will not alter performance

✅ 390 untouched benchmarks
🆕 3 new benchmarks
⏩ 115 skipped benchmarks¹

Performance Changes

	Benchmark	`BASE`	`HEAD`	Efficiency
🆕	`test_claude_model_name_with_noreply_email`	N/A	215.9 µs	N/A
🆕	`test_human_name_with_ordinal_ignored`	N/A	164.5 µs	N/A
🆕	`test_space_separated_model_name_detected`	N/A	205.5 µs	N/A

_{Comparing stasimus:fix-space-separated-model-names (94c2a23) with main (91c4a9f)²}

115 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
No successful run was found on main (b75ec1c) during the generation of this report, so 91c4a9f was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

stasimus requested a review from a team as a code owner July 31, 2026 21:54

stasimus requested review from shenxianpeng and removed request for a team July 31, 2026 21:54

github-actions Bot added bug Something isn't working tests Add test related changes labels Jul 31, 2026

coderabbitai Bot reviewed Jul 31, 2026

View reviewed changes

Comment thread commit_check/ai_signatures_data.py

sshevchenko and others added 2 commits August 1, 2026 01:13

fix: detect space-separated AI model names in trailers

97ebd3e

ci: auto fixes from pre-commit.com hooks

94c2a23

shenxianpeng force-pushed the fix-space-separated-model-names branch from 8fd6c1a to 94c2a23 Compare July 31, 2026 22:13

shenxianpeng approved these changes Jul 31, 2026

View reviewed changes

shenxianpeng removed the tests Add test related changes label Jul 31, 2026

shenxianpeng merged commit d534a3f into commit-check:main Jul 31, 2026
29 checks passed

Navigation Menu

Uh oh!

Uh oh!

fix: detect space-separated AI model names in co-author trailers - #506

fix: detect space-separated AI model names in co-author trailers#506
shenxianpeng merged 2 commits into
commit-check:mainfrom
stasimus:fix-space-separated-model-names

stasimus commented Jul 31, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

netlify Bot commented Jul 31, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Jul 31, 2026 •

edited

Loading

Walkthrough

Changes

Uh oh!

sonarqubecloud Bot commented Jul 31, 2026

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

sonarqubecloud Bot commented Jul 31, 2026

Uh oh!

codecov Bot commented Jul 31, 2026

Uh oh!

shenxianpeng left a comment

Uh oh!

codspeed-hq Bot commented Jul 31, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

stasimus commented Jul 31, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

netlify Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for commit-check ready!

Uh oh!

coderabbitai Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

sonarqubecloud Bot commented Jul 31, 2026

Quality Gate passed

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sonarqubecloud Bot commented Jul 31, 2026

Quality Gate passed

Uh oh!

codecov Bot commented Jul 31, 2026

Codecov Report

Uh oh!

shenxianpeng left a comment

Choose a reason for hiding this comment

Uh oh!

codspeed-hq Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Performance Changes

Footnotes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

stasimus commented Jul 31, 2026 •

edited by coderabbitai Bot

Loading

netlify Bot commented Jul 31, 2026 •

edited

Loading

coderabbitai Bot commented Jul 31, 2026 •

edited

Loading

codspeed-hq Bot commented Jul 31, 2026 •

edited

Loading