Skip to content

JS-1764 Re-enable rspec-maven-plugin for rule data generation #7059

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
zglicz wants to merge 13 commits into
base: master
Choose a base branch
from
Open

JS-1764 Re-enable rspec-maven-plugin for rule data generation #7059

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
2a8c219
Re-enable rspec-maven-plugin for rule data generation
zglicz May 19, 2026
98830dd
Bootstrap api before standalone rspec generation
zglicz May 19, 2026
5359596
Optimize CI rspec rule-data generation
zglicz May 19, 2026
8f298ca
Fix rspec rule-data artifact extraction path
zglicz May 19, 2026
efacf0d
Fix generate-meta wrapper on Windows
zglicz May 19, 2026
da9056a
Remove obsolete JS rspec sync script
zglicz May 19, 2026
69a338e
Reuse prepared RSPEC rule data in builds
zglicz May 19, 2026
0476b4f
Reuse RSPEC data when outputs already exist
zglicz May 19, 2026
c8b9ba7
Fix RSPEC prepared-data reuse
zglicz May 19, 2026
c53ce74
Fix RSPEC prepare command
zglicz May 19, 2026
abe6bdc
Fix RSPEC artifact extraction path
zglicz May 19, 2026
4278583
Skip RSPEC prepare for fork PRs
zglicz May 20, 2026
ea5dd33
Clarify local RSPEC token setup
zglicz May 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
138 changes: 52 additions & 86 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,10 +140,10 @@ jobs:
permissions: *read_permissions
steps: *populate_npm_cache_steps

sync_rspec:
runs-on: sonar-m
name: Sync RSPEC metadata
needs: [setup, populate_npm_cache]
build:
runs-on: sonar-l
name: Build SonarJS on Linux
needs: [setup, get_build_number, populate_npm_cache, prepare_rspec_rule_data]
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork == false
permissions: *read_permissions
steps:
Expand All @@ -164,39 +164,12 @@ jobs:
with:
path: node_modules
key: npm-${{ runner.os }}-${{ needs.setup.outputs.npm-hash }}
- &rspec_secrets
id: rspec-secrets
uses: SonarSource/vault-action-wrapper@v3
with:
secrets: |
development/github/token/{REPO_OWNER_NAME_DASH}-rspec token | RSPEC_GITHUB_TOKEN;
- name: Sync RSPEC rule data
run: npm run sync-rspec-all
env:
GITHUB_TOKEN: ${{ fromJSON(steps.rspec-secrets.outputs.vault).RSPEC_GITHUB_TOKEN }}
- name: Upload RSPEC rule data
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
- &download_rspec_rule_data
name: Download prepared RSPEC rule data
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: rspec-rule-data
path: |
sonar-plugin/javascript-checks/src/main/resources/org/sonar/l10n/javascript/rules/javascript
sonar-plugin/css/src/main/resources/org/sonar/l10n/css/rules/css
resources/rule-data/javascript
resources/rule-data/css
resources/rule-data/.synced-sha-javascript
resources/rule-data/.synced-sha-css
retention-days: 1

build:
runs-on: sonar-l
name: Build SonarJS on Linux
needs: [setup, get_build_number, populate_npm_cache, sync_rspec]
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork == false
permissions: *read_permissions
steps:
- *checkout
- *mise
- *npm_cache
name: rspec-rule-data-${{ github.sha }}
path: sonar-plugin
- &maven_cache
uses: ./.github/actions/maven-cache
with:
Expand All @@ -209,12 +182,6 @@ jobs:
with:
artifactory-reader-role: private-reader
disable-caching: 'true'
- &download_rspec_rule_data
name: Download RSPEC rule data
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: rspec-rule-data
- *rspec_secrets
- id: deployer-secrets
uses: SonarSource/vault-action-wrapper@v3
with:
Expand All @@ -231,7 +198,6 @@ jobs:
ARTIFACTORY_DEPLOY_REPO: sonarsource-public-qa
SIGN_KEY: ${{ fromJSON(steps.deployer-secrets.outputs.vault).SIGN_KEY }}
PGP_PASSPHRASE: ${{ fromJSON(steps.deployer-secrets.outputs.vault).PGP_PASSPHRASE }}
GITHUB_TOKEN: ${{ fromJSON(steps.rspec-secrets.outputs.vault).RSPEC_GITHUB_TOKEN }}
- name: Upload SonarJS artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
Expand Down Expand Up @@ -262,25 +228,53 @@ jobs:
build_win:
runs-on: github-windows-latest-m
name: Build SonarJS on Windows
needs: [setup, populate_npm_cache_win, sync_rspec]
needs: [setup, populate_npm_cache_win, prepare_rspec_rule_data]
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork == false
permissions: *read_permissions
steps:
- *checkout
- *mise
- *npm_cache
- *download_rspec_rule_data
- *maven_cache
- *config_maven
- *download_rspec_rule_data
- *rspec_secrets
- name: Build and test Maven (no deploy)
run: mvn verify -T1C

prepare_rspec_rule_data:
runs-on: sonar-xs
name: Prepare RSPEC rule data
needs: [setup, populate_npm_cache]
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork == false
permissions: *read_permissions
Comment thread
sonar-review-alpha[bot] marked this conversation as resolved.
steps:
- *checkout
- *mise
- *npm_cache
- *maven_cache
- *config_maven
- id: rspec-secrets
uses: SonarSource/vault-action-wrapper@v3
with:
secrets: |
development/github/token/{REPO_OWNER_NAME_DASH}-rspec token | RSPEC_GITHUB_TOKEN;
- name: Generate and deploy RSPEC rule data
run: npm run generate-rule-data:maven
env:
GITHUB_TOKEN: ${{ fromJSON(steps.rspec-secrets.outputs.vault).RSPEC_GITHUB_TOKEN }}
- name: Upload prepared RSPEC rule data
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: rspec-rule-data-${{ github.sha }}
path: |
sonar-plugin/javascript-checks/src/main/resources/org/sonar/l10n/javascript/rules/javascript
sonar-plugin/javascript-checks/src/main/resources/rspec.sha
sonar-plugin/css/src/main/resources/org/sonar/l10n/css/rules/css
retention-days: 1

build_eslint_plugin:
runs-on: github-ubuntu-latest-s
needs: [setup, populate_npm_cache, sync_rspec]
needs: [setup, prepare_rspec_rule_data]
name: Build ESLint Plugin
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork == false
permissions: *read_permissions
Expand All @@ -297,13 +291,9 @@ jobs:
run: |
npm config set //repox.jfrog.io/artifactory/api/npm/:_authToken=${{ fromJSON(steps.secrets.outputs.vault).ARTIFACTORY_ACCESS_TOKEN }}
npm config set registry https://repox.jfrog.io/artifactory/api/npm/npm/
- *npm_cache
- *download_rspec_rule_data
- *rspec_secrets
- name: Build ESLint plugin
run: npm run eslint-plugin:build
env:
GITHUB_TOKEN: ${{ fromJSON(steps.rspec-secrets.outputs.vault).RSPEC_GITHUB_TOKEN }}
- &eslint_tarball_cache
name: Cache ESLint plugin tarball
uses: SonarSource/gh-action_cache@v1
Expand All @@ -313,7 +303,7 @@ jobs:

eslint_readme_freshness:
runs-on: github-ubuntu-latest-s
needs: [setup, populate_npm_cache, sync_rspec]
needs: [setup, populate_npm_cache, prepare_rspec_rule_data]
name: ESLint README Freshness
if: github.event_name == 'schedule'
permissions:
Expand All @@ -335,12 +325,8 @@ jobs:
npm config set registry https://repox.jfrog.io/artifactory/api/npm/npm/
- *npm_cache
- *download_rspec_rule_data
- *rspec_secrets
- name: Regenerate README files
env:
GITHUB_TOKEN: ${{ fromJSON(steps.rspec-secrets.outputs.vault).RSPEC_GITHUB_TOKEN }}
run: |
git config --global url."https://x-access-token:${GITHUB_TOKEN}@github.com/".insteadOf "git@github.com:"
npm install --no-save builtin-modules@3.3.0
npm run eslint-plugin:compile
npm run count-rules
Expand Down Expand Up @@ -395,25 +381,22 @@ jobs:
knip:
runs-on: sonar-xs
name: Knip
needs: [setup, populate_npm_cache, sync_rspec]
needs: [setup, populate_npm_cache, prepare_rspec_rule_data]
permissions: *read_permissions
steps:
- *checkout
- *mise
- *npm_cache
- *download_rspec_rule_data
- *rspec_secrets
- name: Run knip
run: |
npm run bbf
npx knip
env:
GITHUB_TOKEN: ${{ fromJSON(steps.rspec-secrets.outputs.vault).RSPEC_GITHUB_TOKEN }}

test_js:
runs-on: sonar-m
name: Unit tests JavaScript/TypeScript
needs: [setup, populate_npm_cache, sync_rspec]
needs: [setup, populate_npm_cache, prepare_rspec_rule_data]
permissions: *read_permissions
steps:
- name: Check JS coverage cache
Expand Down Expand Up @@ -443,29 +426,22 @@ jobs:
path: node_modules
key: npm-${{ runner.os }}-${{ needs.setup.outputs.npm-hash }}
- if: steps.cache.outputs.cache-hit != 'true'
name: Download RSPEC rule data
name: Download prepared RSPEC rule data
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: rspec-rule-data
- if: steps.cache.outputs.cache-hit != 'true'
id: rspec-secrets
uses: SonarSource/vault-action-wrapper@v3
with:
secrets: |
development/github/token/{REPO_OWNER_NAME_DASH}-rspec token | RSPEC_GITHUB_TOKEN;
name: rspec-rule-data-${{ github.sha }}
path: sonar-plugin
- if: steps.cache.outputs.cache-hit != 'true'
name: Run JS tests with coverage
run: |
npm run generate-meta
npm run bridge:compile
npm run bridge:test:cov
env:
GITHUB_TOKEN: ${{ fromJSON(steps.rspec-secrets.outputs.vault || '{}').RSPEC_GITHUB_TOKEN }}

test_js_win:
runs-on: github-windows-latest-m
name: Unit tests JavaScript on Windows
needs: [setup, populate_npm_cache_win, sync_rspec]
needs: [setup, populate_npm_cache_win, prepare_rspec_rule_data]
permissions: *read_permissions
steps:
- name: Cache JS test results marker (Windows)
Expand Down Expand Up @@ -495,16 +471,11 @@ jobs:
path: node_modules
key: npm-${{ runner.os }}-${{ needs.setup.outputs.npm-hash }}
- if: steps.cache.outputs.cache-hit != 'true'
name: Download RSPEC rule data
name: Download prepared RSPEC rule data
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: rspec-rule-data
- if: steps.cache.outputs.cache-hit != 'true'
id: rspec-secrets
uses: SonarSource/vault-action-wrapper@v3
with:
secrets: |
development/github/token/{REPO_OWNER_NAME_DASH}-rspec token | RSPEC_GITHUB_TOKEN;
name: rspec-rule-data-${{ github.sha }}
path: sonar-plugin
- if: steps.cache.outputs.cache-hit != 'true'
name: Run JS tests on Windows
shell: bash
Expand All @@ -513,8 +484,6 @@ jobs:
npm run bridge:compile
npm run bridge:test:js
mkdir -p .js-test-marker-win && touch .js-test-marker-win/success
env:
GITHUB_TOKEN: ${{ fromJSON(steps.rspec-secrets.outputs.vault || '{}').RSPEC_GITHUB_TOKEN }}

analyze_primary:
runs-on: sonar-m
Expand Down Expand Up @@ -1017,7 +986,7 @@ jobs:
js_ts_ruling:
runs-on: sonar-xl
name: JS/TS Ruling
needs: [setup, populate_npm_cache, sync_rspec]
needs: [setup, populate_npm_cache, prepare_rspec_rule_data]
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork == false
permissions:
id-token: write
Expand All @@ -1032,14 +1001,11 @@ jobs:
- *mise
- *npm_cache
- *download_rspec_rule_data
- *rspec_secrets
- name: Run JS/TS Ruling
id: ruling
run: |
npm run generate-meta
npm run ruling
env:
GITHUB_TOKEN: ${{ fromJSON(steps.rspec-secrets.outputs.vault).RSPEC_GITHUB_TOKEN }}
- name: Update ruling and notify
if: always() && (github.event_name == 'pull_request' || github.ref_name == github.event.repository.default_branch)
env:
Expand Down
8 changes: 8 additions & 0 deletions .github/workflows/docker-a3s-repox.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,16 @@ jobs:
version: 2025.11.2
mise_toml: |
[tools]
java = "21.0"
maven = "3.9"
node = "24.11.0"

- name: Configure Maven
uses: SonarSource/ci-github-actions/config-maven@master
with:
artifactory-reader-role: private-reader
disable-caching: 'true'

- uses: SonarSource/ci-github-actions/config-npm@v1

- name: Access vault secrets
Expand Down
8 changes: 8 additions & 0 deletions .github/workflows/docker-a3s.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,16 @@ jobs:
version: 2025.11.2
mise_toml: |
[tools]
java = "21.0"
maven = "3.9"
node = "24.11.0"

- name: Configure Maven
uses: SonarSource/ci-github-actions/config-maven@master
with:
artifactory-reader-role: private-reader
disable-caching: 'true'

- uses: SonarSource/ci-github-actions/config-npm@v1

- name: Access vault secrets
Expand Down
14 changes: 12 additions & 2 deletions .github/workflows/release_eslint_plugin.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,19 @@ jobs:
with:
jfrogAccessToken: ${{ fromJSON(steps.secrets.outputs.vault).qa_deployer_access_token }}
- uses: actions/checkout@v6
- uses: actions/setup-node@v6
- uses: jdx/mise-action@v4.0.0
with:
node-version: 24
version: 2025.11.2
mise_toml: |
[tools]
java = "21.0"
maven = "3.9"
node = "24.11.0"
- name: Configure Maven
uses: SonarSource/ci-github-actions/config-maven@master
with:
artifactory-reader-role: private-reader
disable-caching: 'true'
- run: cp .github/workflows/.npmrc .npmrc
- name: Publish npm package
env:
Expand Down
4 changes: 3 additions & 1 deletion docs/BUILD.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
Run `npm ci` first on a fresh checkout, and again after any `package.json` or lockfile change.

Use `mvn install` for normal development after Node dependencies are installed.
When generated RSPEC outputs are already present, this reuses them instead of refetching RSPEC.

Avoid `mvn clean` while iterating. The fast Java-only loop reuses previously generated artifacts, and `clean` deletes them. Only use `mvn clean install` when you explicitly want to rebuild generated assets from scratch.

Expand Down Expand Up @@ -92,7 +93,7 @@ It skips:

Important details:

- `npm run generate-meta` already includes RSPEC sync and JS/TS proto generation, so those are skipped too.
- `npm run generate-meta` reuses prepared RSPEC outputs when they already exist, and refreshes them when they do not.
- The `bridge` module still adds `target/generated-sources` to the Java source roots, so an existing generated stub directory can be reused without re-running protobuf generation.
- This flag is intended for Java-only loops after a previous non-skipped build.

Expand Down Expand Up @@ -125,6 +126,7 @@ The clean phase removes the outputs that make the fast loop work:
- `sonar-plugin/sonar-javascript-plugin/src/main/resources/node-info.properties`
- `lib/` and `bin/`
- downloaded rule data under `resources/rule-data`
- generated rule data copied under `sonar-plugin/javascript-checks/src/main/resources` and `sonar-plugin/css/src/main/resources`
- generated rule metadata under `packages/analysis/src/jsts/rules`

Because of that, a common workflow is:
Expand Down
Loading
Loading