Skip to content

JS-1764 Re-enable rspec-maven-plugin for rule data generation#7059

Open
zglicz wants to merge 13 commits into
masterfrom
codex/reenable-rspec-maven-plugin
Open

JS-1764 Re-enable rspec-maven-plugin for rule data generation#7059
zglicz wants to merge 13 commits into
masterfrom
codex/reenable-rspec-maven-plugin

Conversation

@zglicz
Copy link
Copy Markdown
Contributor

@zglicz zglicz commented May 19, 2026

Summary

  • re-enable rspec-maven-plugin for JS/CSS rule-data generation and make npm run generate-meta invoke the Maven-backed RSPEC refresh only when the generated JS/CSS outputs or rspec.sha are missing
  • remove the obsolete JS sync-rspec HTML/JSON generation path and drop the now-unused @asciidoctor/core dependency
  • make Maven reuse existing generated RSPEC outputs automatically, so local mvn install reuses them and mvn clean install refreshes them
  • make CI prepare rule data once, publish it as an artifact, and reuse it from downstream jobs without a dedicated skip env var
  • keep the artifact download target at sonar-plugin/, because the uploaded zip strips that leading path and needs to be re-expanded there to recreate sonar-plugin/javascript-checks/... and sonar-plugin/css/...
  • make generate-rule-data:maven bootstrap sonar-plugin/api first, then run the javascript-checks lifecycle with RSPEC generation enabled
  • make deploy-rule-data publish sonar-plugin/javascript-checks/src/main/resources/rspec.sha from the rule-api cache clone so the prepared artifact is complete
  • include sonar-plugin/javascript-checks/src/main/resources/rspec.sha in the prepared artifact and make clean remove the generated JS/CSS plugin resource copies as part of the refresh path
  • update release/docker workflows and developer docs for the Maven-backed RSPEC flow and token usage

Validation

  • clean regeneration path:
    • rm -rf sonar-plugin/javascript-checks/src/main/resources/org/sonar/l10n/javascript/rules/javascript sonar-plugin/css/src/main/resources/org/sonar/l10n/css/rules/css sonar-plugin/javascript-checks/src/main/resources/rspec.sha resources/rule-data sonar-plugin/javascript-checks/target
    • GITHUB_TOKEN="$(gh auth token 2>/dev/null)" npm run generate-rule-data:maven
    • verified this recreated the JS rule-data folder, the CSS rule-data folder, and sonar-plugin/javascript-checks/src/main/resources/rspec.sha
  • prepared-data fast path:
    • env -u GITHUB_TOKEN npm run generate-meta
    • verified this reused prepared outputs and ran only generate-meta:raw
  • downstream Maven reuse:
    • env -u GITHUB_TOKEN mvn -B -ntp -pl sonar-plugin/sonar-javascript-plugin -am -DskipTests package
    • verified there were no generate-rspec, generate-rule-data, deploy-rule-data, or org.sonarsource.rspec executions in the Maven log
  • jar contents:
    • verified the produced plugin jar contains org/sonar/l10n/javascript/rules/javascript/**, org/sonar/l10n/css/rules/css/**, and rspec.sha
    • expected counts: 531 JS json, 528 JS html, 32 CSS json, 30 CSS html
  • artifact layout:
    • downloaded the prepared artifact from GitHub Actions and verified that extracting it under sonar-plugin/ recreates the expected sonar-plugin/javascript-checks/..., sonar-plugin/css/..., and sonar-plugin/javascript-checks/src/main/resources/rspec.sha paths
  • parsed .github/workflows/build.yml successfully after the workflow refactor

Notes

  • rule-api did not need a companion change for this branch; the current plugin already reads GITHUB_TOKEN
  • on a fresh checkout, or after mvn clean, local RSPEC refresh still needs GITHUB_TOKEN
  • there is a separate pre-existing workflow failure on QA on Windows (2) around .github/workflows/build.yml line 931 / fromJSON(...licenses_token); that also reproduces on master

@hashicorp-vault-sonar-prod hashicorp-vault-sonar-prod Bot changed the title Re-enable rspec-maven-plugin for rule data generation JS-1764 Re-enable rspec-maven-plugin for rule data generation May 19, 2026
@hashicorp-vault-sonar-prod
Copy link
Copy Markdown

hashicorp-vault-sonar-prod Bot commented May 19, 2026

JS-1764

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 19, 2026

Ruling Report

No changes to ruling expected issues in this PR

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented May 19, 2026

Agentic Analysis: Early Results

Agentic Analysis and Context Augmentation are available on your project. Here are some issues that could have been prevented. Follow the links to learn how to put them into action.

1 issue(s) found across 1 file(s):

Rule File Line Message
javascript:S3358 tools/generate-meta.mjs 65 Extract this nested ternary operation into an independent statement.

Analyzed by SonarQube Agentic Analysis in 2.2 s

@zglicz zglicz marked this pull request as ready for review May 19, 2026 15:58
@sonar-review-alpha
Copy link
Copy Markdown
Contributor

sonar-review-alpha Bot commented May 19, 2026

Summary

This PR replaces the Node-based RSPEC sync path with Maven-backed rule data generation. Key changes:

Generation orchestration: New tools/generate-meta.mjs script checks for pre-prepared RSPEC outputs (JS/CSS rule data + rspec.sha). If present, it skips Maven and runs metadata generation only—enabling fast local iteration. If missing, it invokes Maven to generate them first.

Maven integration: The rspec-maven-plugin is now enabled in sonar-plugin/javascript-checks/pom.xml with three auto-activating profiles. Each profile enables RSPEC generation only when its corresponding output file is missing, allowing Maven to reuse existing outputs on subsequent builds.

CI efficiency: Workflow refactored from per-job RSPEC sync to a centralized prepare_rspec_rule_data job that runs once per commit, publishes the artifact, and is reused by all downstream jobs (build, tests, eslint plugin, etc.). Eliminates duplicate token plumbing and token usage.

Removed: Old sync-rspec.ts path (682 lines), associated tools, and the now-unused @asciidoctor/core dependency.

Local developer experience: Developers must now set GITHUB_TOKEN for fresh checkouts or after mvn clean. The old --rspec-path workaround no longer works, but the new flow is simpler and more consistent with CI.

What reviewers should know

Start here: Look at tools/generate-meta.mjs to understand the orchestration logic. It's the decision point that determines whether Maven runs.

Understand the Maven profiles: In sonar-plugin/javascript-checks/pom.xml, the three profiles at the bottom use file-existence activation (<missing> tags) to auto-enable RSPEC generation. This is elegant but important—Maven will skip the expensive plugin execution if outputs already exist.

Workflow refactoring: .github/workflows/build.yml shows the big picture. Find the new prepare_rspec_rule_data job and trace how its artifact (rspec-rule-data-${{ github.sha }}) is downloaded by other jobs. Notice that the rspec-secrets step is now only in one place.

Artifact layout: The uploaded artifact is structured as sonar-plugin/... to match the final directory structure. When extracted, it recreates the exact paths the Maven build expects.

Validation to spot-check: The author's validation steps in the description cover the main scenarios—clean regeneration, prepared-data reuse, and downstream Maven reuse. These are good regression tests.

Known gotcha: Each commit gets its own artifact via github.sha. This prevents accidental reuse across commits but means the artifact cannot be manually promoted between branches.

Breaking change for developers: GITHUB_TOKEN is now required for local builds. The old --rspec-path flag no longer exists. Update onboarding docs accordingly.


  • Generate Walkthrough
  • Generate Diagram

🗣️ Give feedback

sonar-review-alpha[bot]

This comment was marked as resolved.

@zglicz zglicz force-pushed the codex/reenable-rspec-maven-plugin branch from 4c45ba5 to abe6bdc Compare May 19, 2026 16:43
@zglicz zglicz temporarily deployed to sca-checking May 19, 2026 16:43 — with GitHub Actions Inactive
sonar-review-alpha[bot]

This comment was marked as outdated.

@zglicz zglicz temporarily deployed to sca-checking May 20, 2026 07:54 — with GitHub Actions Inactive
sonar-review-alpha[bot]

This comment was marked as outdated.

@zglicz zglicz temporarily deployed to sca-checking May 20, 2026 08:08 — with GitHub Actions Inactive
Copy link
Copy Markdown
Contributor

@sonar-review-alpha sonar-review-alpha Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! ✅

🗣️ Give feedback

@sonarqube-next
Copy link
Copy Markdown

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
0 Dependency risks
No data about Coverage
No data about Duplication

See analysis details on SonarQube

@zglicz zglicz requested a review from a team May 20, 2026 08:27
Copy link
Copy Markdown
Contributor

@vdiez vdiez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dug into why sync-rspec.ts existed on master, and I do not think this PR is functionally equivalent yet for local developer workflows.

Historical context first.

The TypeScript sync-rspec.ts tooling was introduced in PR #6492 (commit c164606ee45891c9174946437d97db99a5b325e3). The point of that change was not just to move logic around. It was added specifically to replace the rspec-maven-plugin path because the Maven flow was not robust enough for day-to-day development:

  • it depended on sonar-rule-api auth behavior that required GITHUB_TOKEN in the environment;
  • it was not as straightforward to run on developer machines as the existing Node-based workflow;
  • SonarJS wanted the generated rule-data that feeds JS/TS builds to be reproducible locally with the normal repo tooling, without special Maven/plugin auth setup.

That history matters here, because the bar for removing sync-rspec.ts is not only “the build passes in CI” or “the generated files look mostly the same”. The replacement also needs to preserve the developer ergonomics we had on master: on a clean machine/worktree, a developer should be able to regenerate the RSPEC-derived outputs with the normal build flow as easily as before.

I tested that explicitly.

On this branch, using a clean worktree, npm ci + npm run bbf does regenerate the expected outputs, and the repository shape is aligned with the intended direction:

  • RSPEC JSON/HTML and rspec.sha are generated at build time and are not git-tracked;
  • generated-meta.ts files are generated at build time and are not git-tracked;
  • generated Java check classes are generated at build time and are not git-tracked.

So the repository-layout goal is mostly correct.

However, there is still one important regression compared to master: if I delete the generated RSPEC outputs in a clean worktree and run the generation flow without exporting GITHUB_TOKEN, the Maven path fails. In practice that means the replacement is still not as easy to run on dev machines as the sync-rspec.ts flow it is removing.

The reason is upstream:

  1. rspec-maven-plugin still does not accept any explicit GitHub auth parameter. The plugin currently only exposes ruleSubdirectory, targetDirectory, and vcsBranchName, and then constructs GitHubRuleMaker.create(branch) internally.
  2. The released plugin version used here (1.3.0.46) depends on sonar-rule-api 2.18.0.5734, where GitHubRuleMaker only looks at System.getenv(GITHUB_TOKEN).
  3. Even on current sonar-rule-api master, while GitHubRuleMaker now falls back to gh auth token, CoverageLoader still hard-requires GITHUB_TOKEN from the environment.

So even if SonarJS switches back to the Maven-based generation flow, we are still missing the pieces needed to make that flow dev-machine-friendly in the same way master was.

For me, the minimum bar for functional equivalence is:

  • a clean checkout/worktree can regenerate the RSPEC-derived outputs without requiring contributors to pre-export GITHUB_TOKEN in their shell;
  • the build can use existing developer auth state (for example gh auth login) or an explicit parameter, instead of only ambient environment variables;
  • the generated HTML/JSON output matches the current TypeScript flow closely enough that we do not introduce content regressions.

Concretely, I think this means the upstream work is still required before SonarJS can safely remove sync-rspec.ts:

  1. In sonar-rule-api, introduce a shared token resolution mechanism used everywhere GitHub access happens.
    Precedence should be something like: explicit token parameter -> GITHUB_TOKEN env var -> gh auth token.
  2. In sonar-rule-api, expose APIs that allow callers to pass the token explicitly instead of forcing env lookup internally (for example GitHubRuleMaker.create(branch, token) and a corresponding CoverageLoader constructor/entry point).
  3. In rspec-maven-plugin, expose an optional auth parameter (or Maven settings-based credential lookup) and pass it explicitly to sonar-rule-api.
  4. After that, SonarJS can bump to the new plugin/rule-api versions and keep this PR’s general repository policy: generated RSPEC outputs, generated TS meta, and generated Java checks should all be build-time artifacts and not be tracked in git.

There is also one output-level issue worth addressing before we treat the Maven flow as a drop-in replacement: I found a real HTML rendering delta on S7724.html where the generated output contains stray <strong> markup around the phrase after /* eslint-disable */. The JSON differences I saw were only ordering/newline noise, but this HTML difference looks substantive.

So my conclusion is:

  • the repository cleanup direction in this PR is good;
  • but the TypeScript sync-rspec.ts tooling was originally added for a concrete developer-workflow reason, not just implementation preference;
  • and we have not fully closed that gap yet, because the Maven path still depends on GITHUB_TOKEN being exported in the environment in scenarios where master was easier to use.

I would be comfortable with this switch once the Maven/plugin/rule-api stack is updated so that local generation works smoothly on developer machines without requiring manual env-token setup, and once the HTML rendering delta is understood/fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants