Skip to content

fix(gem): discover flat BUNDLE_PATH gem layouts — bundler-1 env-var installs were invisible to scan/get/apply - #218

Merged
Mikola Lysenko (mikolalysenko) merged 1 commit into
mainfrom
fix/gem-crawler-flat-bundle-path
Aug 19, 2026
Merged

fix(gem): discover flat BUNDLE_PATH gem layouts — bundler-1 env-var installs were invisible to scan/get/apply#218
Mikola Lysenko (mikolalysenko) merged 1 commit into
mainfrom
fix/gem-crawler-flat-bundle-path

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Defect (D1, gem live-matrix campaign 2026-08-19 — adversarially verified 2/2 in fresh containers)

Bundler 1.x with BUNDLE_PATH set via the environment variable installs GEM_HOME-style into the flat <BUNDLE_PATH>/gems/ — no <engine>/<abi> scope segment, sibling specifications/ dir present. The ruby crawler only enumerated the scoped vendor/bundle/<engine>/<abi>/gems layout, so on such projects:

  • scan --json reported the installed gem "notInstalled": true (lockfile-only), and
  • get <purl> returned partial_failure downloaded 1 / applied 0 (package_not_installed) — the patch never landed, while the gem sat on disk at vendor/bundle/gems/activestorage-6.0.3.

The gem env fallback missed it too (the flat store is not a gem home gem env reports). Verified live against bundler 1.17.3 / ruby 3.1.7 with activestorage@6.0.3.

Root cause

RubyCrawler::get_vendor_bundle_paths hardcoded the scoped shape: it walked vendor/bundle/<engine>/<version>/gems only. Bundler.bundle_path on bundler 1 appends ruby_scope only for --path/local-config installs; an env-var BUNDLE_PATH is used verbatim as a GEM_HOME, producing the flat layout. bundler >= 2 appends the scope in both styles, so the gap is bundler-1-env-specific — but the flat layout is also plain GEM_HOME semantics, so it can legitimately appear under any explicit bundle root.

Layout matrix

install style bundler 1 bundler 2 bundler 4
--path / bundle config set --local path <root>/<engine>/<abi>/gems (scoped) scoped scoped (--path removed; config/env only)
env BUNDLE_PATH <root>/gems (FLAT) — was invisible scoped scoped

(<engine>/<abi> = Bundler.ruby_scope, e.g. ruby/3.1.0; flat roots carry a sibling specifications/ dir.)

Fix

All of scan/get/apply flow through get_gem_paths, so one fix heals all three (crates/socket-patch-core/src/crawlers/ruby_crawler.rs):

  • Probe each bundler install root in BOTH layouts: the scoped <root>/<engine>/<abi>/gems walk as before, plus flat <root>/gems guarded on the sibling specifications/ directory every real gem home carries — a random gems/ dir is not mistaken for a gem store.
  • Honor an explicit BUNDLE_PATH install root: the env var, and the BUNDLE_PATH: entry of bundler's app config file ($BUNDLE_APP_CONFIG/config, else <cwd>/.bundle/config — reusing setup::gem::bundler_app_config_dir, now pub(crate)). Explicit roots are gated on the cwd holding a Bundler manifest (same "looks like a Ruby project" gate as the gem env fallback), so a machine-wide BUNDLE_PATH export cannot pull another project's gem store into a non-Ruby scan. Relative values resolve against the project root, matching Bundler.bundle_path.
  • Dedup roots and discovered gems/ dirs (env var naming vendor/bundle doesn't scan twice), and skip a flat store's gems/ entry in the engine walk so a gem shipping its own gems/ subdir can't surface a ghost <engine>/<version>/gems root.
  • The config-file scrape is a line-based parser (parse_bundle_config_path), matching the repo's Cargo.toml line-parsing convention; env reads live in a thin wrapper over get_vendor_bundle_paths_with_env so tests stay hermetic.

Tests (test-first: RED before the fix, GREEN after)

RED run reproduced the defect exactly — flat store invisible, coexist walk surfacing the ghost weird-1.0.0/gems engine path, and crawl_all falling through to the host's gem env homes instead of the project store.

New unit tests in ruby_crawler.rs:

  • get_vendor_bundle_paths_flat_bundler1_layout — flat vendor/bundle/gems + specifications/ discovered (RED before fix)
  • get_vendor_bundle_paths_ignores_bare_gems_dir — bare gems/ without specifications/ rejected
  • get_vendor_bundle_paths_scoped_and_flat_coexist — both layouts under one root, exactly once, ghost-engine guard (RED)
  • crawl_all_finds_flat_bundler1_project — full local-mode pipeline on the repro shape (RED)
  • bundle_path_env_discovers_both_layouts, bundle_path_env_relative_resolves_against_project_root, bundle_path_env_ignored_without_manifest, bundle_path_env_duplicate_root_dedups
  • app_config_bundle_path_discovered, app_config_env_relocates_config, parse_bundle_config_path_contract

Gates:

  • cargo test -p socket-patch-core --lib crawlers::ruby — 40 passed
  • cargo test -p socket-patch-core --lib — 2393 passed, 0 failed
  • cargo test -p socket-patch-cli --test e2e_gem — 8 passed, 3 ignored (hermetic suite)
  • cargo clippy -p socket-patch-core --lib --tests -- -D warnings — clean
  • cargo fmt --check — clean on touched files

Live proof (fixed binary, campaign container)

Re-ran the campaign repro in socket-patch-test-gem-b1:gemx (ruby 3.1.7, bundler 1.17.3) with the fixed binary (linux arm64 release build of this branch) bind-mounted over /usr/local/bin/socket-patch — real bundle install with env BUNDLE_PATH, real scan/get against production:

OK(scan): activestorage not flagged notInstalled
OK(get): "applied": 1 "status": "success"
OK(disk): patch marker present in vendor/bundle/gems/activestorage-6.0.3/lib/active_storage/service/s3_service.rb

All three defect checks that failed 2/2 pre-fix now pass.

Campaign repro

scratchpad/gemx/lanes/agent-b1/run-bundle-path-env-layout.sh (gem live-matrix 2026-08-19, image socket-patch-test-gem-b1:gemx: ruby 3.1.7, bundler 1.17.3) — exits non-zero while the defect is present: DEFECT(scan) + DEFECT(get/apply) + DEFECT(disk).

🤖 Generated with Claude Code


Note

Medium Risk
Changes local gem path discovery used by scan/get/apply; incorrect paths could miss or mis-target patches, but behavior is heavily unit-tested and explicit roots are gated on Bundler manifests.

Overview
Fixes gem scan/get/apply missing gems when Bundler installs into a flat gems/ tree (Bundler 1 with env BUNDLE_PATH) instead of only the scoped vendor/bundle/<engine>/<abi>/gems layout.

The Ruby crawler now probes each Bundler install root in both layouts: the existing scoped walk plus flat <root>/gems when a sibling specifications/ dir exists, with deduping and a guard so a flat store’s gems/ entry isn’t treated as an engine directory.

Install roots are no longer just vendor/bundle: env BUNDLE_PATH and the BUNDLE_PATH: line in Bundler app config (via shared bundler_app_config_dir, now pub(crate)) are honored when the cwd has a Bundler manifest; relative paths resolve against the project root. Explicit roots stay gated so a global BUNDLE_PATH doesn’t pollute non-Ruby scans.

New helpers resolve_bundle_path and parse_bundle_config_path plus a large unit-test suite cover flat layout, coexistence with scoped layout, env/config paths, and the full crawl_all path.

Reviewed by Cursor Bugbot for commit 698701c. Configure here.

…nstalls were invisible to scan/get/apply

The ruby crawler only enumerated the scoped bundler layout
vendor/bundle/<engine>/<abi>/gems. Bundler 1.x with BUNDLE_PATH set via
the ENVIRONMENT installs GEM_HOME-style into the flat <BUNDLE_PATH>/gems
(no ruby_scope segment; sibling specifications/ dir present), so on such
projects `scan --json` reported installed gems "notInstalled": true and
`get <purl>` returned partial_failure downloaded 1 / applied 0 — the
patch never landed while the gem sat on disk. Live-verified 2/2 in
fresh containers 2026-08-19 (gem live-matrix D1: activestorage@6.0.3,
bundler 1.17.3, vendor/bundle/gems/activestorage-6.0.3).

Layout matrix bundler actually produces:

  install style        bundler 1        bundler 2/4
  --path / local cfg   <root>/<scope>   <root>/<scope>
  env BUNDLE_PATH      <root>  (FLAT)   <root>/<scope>

  (<scope> = <engine>/<abi>, e.g. ruby/3.1.0)

Fix — all of scan/get/apply flow through get_gem_paths, so one fix
heals all three:

- probe each bundler install root in BOTH layouts: the scoped
  <root>/<engine>/<abi>/gems walk as before, plus flat <root>/gems
  guarded on the sibling specifications/ dir every real gem home
  carries (no false positives on random gems/ dirs);
- honor an explicit BUNDLE_PATH install root — env var, and the
  BUNDLE_PATH: entry of the app config file ($BUNDLE_APP_CONFIG/config,
  else .bundle/config; reuses setup::gem::bundler_app_config_dir) —
  gated on the cwd holding a Bundler manifest, the same "looks like a
  Ruby project" gate as the gem-env fallback; relative values resolve
  against the project root like Bundler.bundle_path;
- dedup roots and discovered gems/ dirs so a root reachable two ways is
  not scanned twice, and skip a flat store's gems/ entry in the engine
  walk so a gem shipping its own gems/ subdir can't surface a ghost
  root.

Tests (test-first, RED before the fix): flat-layout discovery, bare
gems/ without specifications/ rejected, scoped+flat coexistence with
dedup and ghost-engine guard, env BUNDLE_PATH both-layout discovery,
relative-path resolution, manifest gating, duplicate-root dedup, app
config discovery incl. relocated $BUNDLE_APP_CONFIG, and the pure
config-scrape parser contract; plus a crawl_all pipeline test shaped
exactly like the live repro.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mikolalysenko
Mikola Lysenko (mikolalysenko) merged commit 3e51259 into main Aug 19, 2026
63 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the fix/gem-crawler-flat-bundle-path branch August 19, 2026 19:17
Mikola Lysenko (mikolalysenko) added a commit that referenced this pull request Aug 20, 2026
…every coexisting copy, contain config-sourced roots

Post-merge audit follow-up on #218 (gem crawler flat-BUNDLE_PATH
discovery), mirroring the #216 npm multi-copy precedent (0433bcb).

1. MULTI-COPY (most severe): bundler's scoped `<engine>/<abi>/gems`
   and flat `gems/` stores coexist under one root, each holding a REAL
   physical copy of the same gem@version — exactly the state #218's
   tests create. First-wins merging resolved the purl to ONE path, so
   apply patched one store and reported success while the other
   bundler loaded pristine (vulnerable) bytes. Fix mirrors #216:
   `find_all_packages_for_purls` now routes the release-variant
   ecosystems through an accumulating `merge_variant_copies` (reusing
   `push_path`, base-PURL keyed, precedence order kept) and apply's
   variant branch fans out per copy for gem — per-copy Applied events,
   `summary.applied` counts each copy, a copy matching no variant
   fails loudly. Rollback already carried every copy via
   `merge_qualified` and its per-path grouping; the new in-process
   suite pins both directions. PyPI/Maven deliberately keep the
   one-representative contract (the nuget `already_patched`
   double-patch regression from #216's second commit), as do all
   collapsing consumers (vendor/vex/setup/get/repair-vendor).
   scan --sync patches every copy too (it runs the real nested apply);
   scan's inventory stays purl-level, byte-identical to npm's
   crawl_all purl-dedup precedent.

2. PRECEDENCE: roots probed local-config > env > default vendor/bundle
   (bundler's real precedence; the old order was inverted), so
   first-representative consumers pick the copy bundler actually loads.

3. REGRESSION vs pre-#218: env/config roots no longer trip the
   `gem env` fallback early-return — only the historic project-local
   vendor/bundle probe keeps it. Default gems (rexml/json) live only
   in the DEFAULT/system gem homes, so an env-BUNDLE_PATH project gets
   those homes appended (deduped) again.

4. SECURITY: a config-sourced BUNDLE_PATH (committed .bundle/config =
   attacker-authored input, and a scan/apply WRITE-target root) must
   now, after ~ expansion and lexical normalization, stay contained in
   the project root — otherwise the root is skipped with a
   `gem_bundle_config_path_ignored` stderr warning naming the value.
   Windows rooted forms (`\evil`, `C:evil`) take the strict branch.
   Env-sourced BUNDLE_PATH stays trusted (user's own environment) but
   is normalized for dedup. `normalize_lexically` is hoisted from the
   composer crawler into utils::fs and shared.

5. `~` EXPANSION: a leading `~`/`~/...` in BUNDLE_PATH expands against
   home (bundler File.expand_path), env-injectable via the _with_env
   seams for hermetic tests.

6. BUNDLE_PATH__SYSTEM: `"true"` makes bundler ignore the recorded
   path — the config entry now parses as unset and the fallback finds
   the system gem homes.

7. TEST HERMETICITY: the six crawler_ruby_e2e tests that read ambient
   BUNDLE_PATH/BUNDLE_APP_CONFIG now route through the new
   `get_gem_paths_with_env` seam; a new e2e pins env-root + gem-env
   fallback coexistence (finding 3).

8. CLI_CONTRACT.md: the stale "gem inspects only <cwd>/vendor/bundle"
   claim replaced with the real root model, the containment policy,
   and the multi-copy behavior.

TDD evidence (red -> green): dispatch-level
`find_all_packages_for_purls_carries_every_gem_store_copy`, crawler
`bundle_roots_probe_in_bundler_precedence_order`, and the new
`in_process_gem_multicopy.rs` (real binary apply/rollback over a
coexisting two-store tree) all failed on base — the flat copy stayed
byte-for-byte VULNERABLE while apply reported success — and pass now.

Live era-image proof (docker run --rm, socket-patch-test-gem-b1:gemx,
bundler 1.17.3): the baked pre-fix binary on a coexist fixture reports
status=success/applied=1 while the flat store's copy — loaded via real
GEM_HOME resolution in the container — still evaluates VULNERABLE; the
fixed binary reports applied=2 and both stores load FIXED.

Gates: touched files rustfmt-clean; cargo clippy --workspace
--all-features -D warnings clean; core --lib 2413 passed; cli --lib
430 passed; crawler_ruby_e2e 25, crawler_composer_e2e 32,
crawlers_empty_paths_e2e 13; e2e_gem hermetic 8; in-process gem+npm
multicopy suites; cli_gem_variant_mismatch_policy 6; docker_e2e_gem,
docker_e2e_vendor_gem, docker_e2e_pypi, docker_e2e_maven all green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mikola Lysenko (mikolalysenko) added a commit that referenced this pull request Aug 20, 2026
…lass-split fallback-home failure semantics

Review-round fixes for the two verified Bugbot findings on #222.

Finding 1 (Medium): the containment guard's
`gem_bundle_config_path_ignored` was a bare eprintln inside the crawler —
it never reached any --json `warnings[]` and printed under --silent,
violating the repo-wide warning conventions (#219/#220 + the #223
omnibus silent fixes). The crawler is now print-free: the refusal is
RECORDED on `BundleStoreDiscovery.skipped_config_path`, and a shared
`config_path_ignored_warning(value)` builder feeds the CLI channels —
scan pushes it onto the same run-level channel as the PnP layout
refusals (JSON `warnings[]` on both the zero-package and >=1-package
envelopes; one stderr line gated on !json && !silent), and apply carries
it in its Envelope `warnings[]` plus one gated stderr line. Scoped like
the crawl that hit it: local mode, gem in --ecosystems/manifest scope.

Finding 2 (High, with nuance): with an env/config bundle root,
get_gem_paths appends the gem-env fallback homes and the multi-copy
fan-out patched EVERY copy with per-copy loud-fail — so a gem present in
both the bundle store and a shared home (rvm @global, root-owned system
dir) failed the WHOLE run on a permission failure or variant mismatch
THERE, even though the copy bundler loads patched fine. Patching a
shared home's copy is not itself wrong (plain apply always patched
GEM_HOME when no store existed); the defect was failure semantics
crossing store classes. Fix: discovery's store list is exposed
(`RubyCrawler::discover_bundle_stores`, fs-probes only) and apply's gem
fan-out classes each copy — bundle-path store copies stay PRIMARY
(loud-fail, unchanged); gem-env fallback-home copies become BEST-EFFORT
once at least one store copy applied: a variant mismatch or write
failure there is a per-copy non-fatal Skipped event
(`gem_fallback_home_skipped`, detail names the path and reason; gated
stderr twin), never a run failure. Parity edge kept: with NO bundle-store
copy (the historic fallback-only layout, and every --global run) the
home copy IS primary and keeps loud-fail exactly as pre-#218 apply.

TDD evidence (red -> green on the rebased tip): scan/apply --json
missing the warnings[] entry and the --silent leak (3 tests,
in_process_gem_config_warning.rs); mismatched-home-copy exit 1 and
Failed-event-on-strict-refusal (in_process_gem_fallback_home.rs), with
fallback-only loud parity and both-copies-patched pinned green
throughout. CLI_CONTRACT.md documents the copy classes and the warning
channels.

Gates: touched files rustfmt-clean; cargo clippy --workspace
--all-features -D warnings clean; core --lib 2535, cli --lib 436;
crawlers::ruby 50, crawler_ruby_e2e 25; gem+npm multicopy 2+2;
apply_network 11, apply_invariants 4, cli_gem_variant_mismatch_policy 6,
cli_apply_silent 2, e2e_gem hermetic 8 (6 shown +cache selftests),
e2e_scan, cli_scan_silent, docker_e2e_gem — all green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mikola Lysenko (mikolalysenko) added a commit that referenced this pull request Aug 20, 2026
…every coexisting copy, contain config-sourced roots (#222)

Post-merge audit follow-up on #218 (gem crawler flat-BUNDLE_PATH
discovery), mirroring the #216 npm multi-copy precedent (0433bcb).

1. MULTI-COPY (most severe): bundler's scoped `<engine>/<abi>/gems`
   and flat `gems/` stores coexist under one root, each holding a REAL
   physical copy of the same gem@version — exactly the state #218's
   tests create. First-wins merging resolved the purl to ONE path, so
   apply patched one store and reported success while the other
   bundler loaded pristine (vulnerable) bytes. Fix mirrors #216:
   `find_all_packages_for_purls` now routes the release-variant
   ecosystems through an accumulating `merge_variant_copies` (reusing
   `push_path`, base-PURL keyed, precedence order kept) and apply's
   variant branch fans out per copy for gem — per-copy Applied events,
   `summary.applied` counts each copy, a copy matching no variant
   fails loudly. Rollback already carried every copy via
   `merge_qualified` and its per-path grouping; the new in-process
   suite pins both directions. PyPI/Maven deliberately keep the
   one-representative contract (the nuget `already_patched`
   double-patch regression from #216's second commit), as do all
   collapsing consumers (vendor/vex/setup/get/repair-vendor).
   scan --sync patches every copy too (it runs the real nested apply);
   scan's inventory stays purl-level, byte-identical to npm's
   crawl_all purl-dedup precedent.

2. PRECEDENCE: roots probed local-config > env > default vendor/bundle
   (bundler's real precedence; the old order was inverted), so
   first-representative consumers pick the copy bundler actually loads.

3. REGRESSION vs pre-#218: env/config roots no longer trip the
   `gem env` fallback early-return — only the historic project-local
   vendor/bundle probe keeps it. Default gems (rexml/json) live only
   in the DEFAULT/system gem homes, so an env-BUNDLE_PATH project gets
   those homes appended (deduped) again.

4. SECURITY: a config-sourced BUNDLE_PATH (committed .bundle/config =
   attacker-authored input, and a scan/apply WRITE-target root) must
   now, after ~ expansion and lexical normalization, stay contained in
   the project root — otherwise the root is skipped with a
   `gem_bundle_config_path_ignored` stderr warning naming the value.
   Windows rooted forms (`\evil`, `C:evil`) take the strict branch.
   Env-sourced BUNDLE_PATH stays trusted (user's own environment) but
   is normalized for dedup. `normalize_lexically` is hoisted from the
   composer crawler into utils::fs and shared.

5. `~` EXPANSION: a leading `~`/`~/...` in BUNDLE_PATH expands against
   home (bundler File.expand_path), env-injectable via the _with_env
   seams for hermetic tests.

6. BUNDLE_PATH__SYSTEM: `"true"` makes bundler ignore the recorded
   path — the config entry now parses as unset and the fallback finds
   the system gem homes.

7. TEST HERMETICITY: the six crawler_ruby_e2e tests that read ambient
   BUNDLE_PATH/BUNDLE_APP_CONFIG now route through the new
   `get_gem_paths_with_env` seam; a new e2e pins env-root + gem-env
   fallback coexistence (finding 3).

8. CLI_CONTRACT.md: the stale "gem inspects only <cwd>/vendor/bundle"
   claim replaced with the real root model, the containment policy,
   and the multi-copy behavior.

TDD evidence (red -> green): dispatch-level
`find_all_packages_for_purls_carries_every_gem_store_copy`, crawler
`bundle_roots_probe_in_bundler_precedence_order`, and the new
`in_process_gem_multicopy.rs` (real binary apply/rollback over a
coexisting two-store tree) all failed on base — the flat copy stayed
byte-for-byte VULNERABLE while apply reported success — and pass now.

Live era-image proof (docker run --rm, socket-patch-test-gem-b1:gemx,
bundler 1.17.3): the baked pre-fix binary on a coexist fixture reports
status=success/applied=1 while the flat store's copy — loaded via real
GEM_HOME resolution in the container — still evaluates VULNERABLE; the
fixed binary reports applied=2 and both stores load FIXED.

Gates: touched files rustfmt-clean; cargo clippy --workspace
--all-features -D warnings clean; core --lib 2413 passed; cli --lib
430 passed; crawler_ruby_e2e 25, crawler_composer_e2e 32,
crawlers_empty_paths_e2e 13; e2e_gem hermetic 8; in-process gem+npm
multicopy suites; cli_gem_variant_mismatch_policy 6; docker_e2e_gem,
docker_e2e_vendor_gem, docker_e2e_pypi, docker_e2e_maven all green.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants