fix(gem): discover flat BUNDLE_PATH gem layouts — bundler-1 env-var installs were invisible to scan/get/apply - #218
Merged
Mikola Lysenko (mikolalysenko) merged 1 commit intoAug 19, 2026
Conversation
…nstalls were invisible to scan/get/apply The ruby crawler only enumerated the scoped bundler layout vendor/bundle/<engine>/<abi>/gems. Bundler 1.x with BUNDLE_PATH set via the ENVIRONMENT installs GEM_HOME-style into the flat <BUNDLE_PATH>/gems (no ruby_scope segment; sibling specifications/ dir present), so on such projects `scan --json` reported installed gems "notInstalled": true and `get <purl>` returned partial_failure downloaded 1 / applied 0 — the patch never landed while the gem sat on disk. Live-verified 2/2 in fresh containers 2026-08-19 (gem live-matrix D1: activestorage@6.0.3, bundler 1.17.3, vendor/bundle/gems/activestorage-6.0.3). Layout matrix bundler actually produces: install style bundler 1 bundler 2/4 --path / local cfg <root>/<scope> <root>/<scope> env BUNDLE_PATH <root> (FLAT) <root>/<scope> (<scope> = <engine>/<abi>, e.g. ruby/3.1.0) Fix — all of scan/get/apply flow through get_gem_paths, so one fix heals all three: - probe each bundler install root in BOTH layouts: the scoped <root>/<engine>/<abi>/gems walk as before, plus flat <root>/gems guarded on the sibling specifications/ dir every real gem home carries (no false positives on random gems/ dirs); - honor an explicit BUNDLE_PATH install root — env var, and the BUNDLE_PATH: entry of the app config file ($BUNDLE_APP_CONFIG/config, else .bundle/config; reuses setup::gem::bundler_app_config_dir) — gated on the cwd holding a Bundler manifest, the same "looks like a Ruby project" gate as the gem-env fallback; relative values resolve against the project root like Bundler.bundle_path; - dedup roots and discovered gems/ dirs so a root reachable two ways is not scanned twice, and skip a flat store's gems/ entry in the engine walk so a gem shipping its own gems/ subdir can't surface a ghost root. Tests (test-first, RED before the fix): flat-layout discovery, bare gems/ without specifications/ rejected, scoped+flat coexistence with dedup and ghost-engine guard, env BUNDLE_PATH both-layout discovery, relative-path resolution, manifest gating, duplicate-root dedup, app config discovery incl. relocated $BUNDLE_APP_CONFIG, and the pure config-scrape parser contract; plus a crawl_all pipeline test shaped exactly like the live repro. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mikola Lysenko (mikolalysenko)
enabled auto-merge (squash)
August 19, 2026 17:29
Wenxin Jiang (Wenxin-Jiang)
approved these changes
Aug 19, 2026
Mikola Lysenko (mikolalysenko)
deleted the
fix/gem-crawler-flat-bundle-path
branch
August 19, 2026 19:17
This was referenced Aug 19, 2026
Mikola Lysenko (mikolalysenko)
added a commit
that referenced
this pull request
Aug 20, 2026
…every coexisting copy, contain config-sourced roots Post-merge audit follow-up on #218 (gem crawler flat-BUNDLE_PATH discovery), mirroring the #216 npm multi-copy precedent (0433bcb). 1. MULTI-COPY (most severe): bundler's scoped `<engine>/<abi>/gems` and flat `gems/` stores coexist under one root, each holding a REAL physical copy of the same gem@version — exactly the state #218's tests create. First-wins merging resolved the purl to ONE path, so apply patched one store and reported success while the other bundler loaded pristine (vulnerable) bytes. Fix mirrors #216: `find_all_packages_for_purls` now routes the release-variant ecosystems through an accumulating `merge_variant_copies` (reusing `push_path`, base-PURL keyed, precedence order kept) and apply's variant branch fans out per copy for gem — per-copy Applied events, `summary.applied` counts each copy, a copy matching no variant fails loudly. Rollback already carried every copy via `merge_qualified` and its per-path grouping; the new in-process suite pins both directions. PyPI/Maven deliberately keep the one-representative contract (the nuget `already_patched` double-patch regression from #216's second commit), as do all collapsing consumers (vendor/vex/setup/get/repair-vendor). scan --sync patches every copy too (it runs the real nested apply); scan's inventory stays purl-level, byte-identical to npm's crawl_all purl-dedup precedent. 2. PRECEDENCE: roots probed local-config > env > default vendor/bundle (bundler's real precedence; the old order was inverted), so first-representative consumers pick the copy bundler actually loads. 3. REGRESSION vs pre-#218: env/config roots no longer trip the `gem env` fallback early-return — only the historic project-local vendor/bundle probe keeps it. Default gems (rexml/json) live only in the DEFAULT/system gem homes, so an env-BUNDLE_PATH project gets those homes appended (deduped) again. 4. SECURITY: a config-sourced BUNDLE_PATH (committed .bundle/config = attacker-authored input, and a scan/apply WRITE-target root) must now, after ~ expansion and lexical normalization, stay contained in the project root — otherwise the root is skipped with a `gem_bundle_config_path_ignored` stderr warning naming the value. Windows rooted forms (`\evil`, `C:evil`) take the strict branch. Env-sourced BUNDLE_PATH stays trusted (user's own environment) but is normalized for dedup. `normalize_lexically` is hoisted from the composer crawler into utils::fs and shared. 5. `~` EXPANSION: a leading `~`/`~/...` in BUNDLE_PATH expands against home (bundler File.expand_path), env-injectable via the _with_env seams for hermetic tests. 6. BUNDLE_PATH__SYSTEM: `"true"` makes bundler ignore the recorded path — the config entry now parses as unset and the fallback finds the system gem homes. 7. TEST HERMETICITY: the six crawler_ruby_e2e tests that read ambient BUNDLE_PATH/BUNDLE_APP_CONFIG now route through the new `get_gem_paths_with_env` seam; a new e2e pins env-root + gem-env fallback coexistence (finding 3). 8. CLI_CONTRACT.md: the stale "gem inspects only <cwd>/vendor/bundle" claim replaced with the real root model, the containment policy, and the multi-copy behavior. TDD evidence (red -> green): dispatch-level `find_all_packages_for_purls_carries_every_gem_store_copy`, crawler `bundle_roots_probe_in_bundler_precedence_order`, and the new `in_process_gem_multicopy.rs` (real binary apply/rollback over a coexisting two-store tree) all failed on base — the flat copy stayed byte-for-byte VULNERABLE while apply reported success — and pass now. Live era-image proof (docker run --rm, socket-patch-test-gem-b1:gemx, bundler 1.17.3): the baked pre-fix binary on a coexist fixture reports status=success/applied=1 while the flat store's copy — loaded via real GEM_HOME resolution in the container — still evaluates VULNERABLE; the fixed binary reports applied=2 and both stores load FIXED. Gates: touched files rustfmt-clean; cargo clippy --workspace --all-features -D warnings clean; core --lib 2413 passed; cli --lib 430 passed; crawler_ruby_e2e 25, crawler_composer_e2e 32, crawlers_empty_paths_e2e 13; e2e_gem hermetic 8; in-process gem+npm multicopy suites; cli_gem_variant_mismatch_policy 6; docker_e2e_gem, docker_e2e_vendor_gem, docker_e2e_pypi, docker_e2e_maven all green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mikola Lysenko (mikolalysenko)
added a commit
that referenced
this pull request
Aug 20, 2026
…lass-split fallback-home failure semantics Review-round fixes for the two verified Bugbot findings on #222. Finding 1 (Medium): the containment guard's `gem_bundle_config_path_ignored` was a bare eprintln inside the crawler — it never reached any --json `warnings[]` and printed under --silent, violating the repo-wide warning conventions (#219/#220 + the #223 omnibus silent fixes). The crawler is now print-free: the refusal is RECORDED on `BundleStoreDiscovery.skipped_config_path`, and a shared `config_path_ignored_warning(value)` builder feeds the CLI channels — scan pushes it onto the same run-level channel as the PnP layout refusals (JSON `warnings[]` on both the zero-package and >=1-package envelopes; one stderr line gated on !json && !silent), and apply carries it in its Envelope `warnings[]` plus one gated stderr line. Scoped like the crawl that hit it: local mode, gem in --ecosystems/manifest scope. Finding 2 (High, with nuance): with an env/config bundle root, get_gem_paths appends the gem-env fallback homes and the multi-copy fan-out patched EVERY copy with per-copy loud-fail — so a gem present in both the bundle store and a shared home (rvm @global, root-owned system dir) failed the WHOLE run on a permission failure or variant mismatch THERE, even though the copy bundler loads patched fine. Patching a shared home's copy is not itself wrong (plain apply always patched GEM_HOME when no store existed); the defect was failure semantics crossing store classes. Fix: discovery's store list is exposed (`RubyCrawler::discover_bundle_stores`, fs-probes only) and apply's gem fan-out classes each copy — bundle-path store copies stay PRIMARY (loud-fail, unchanged); gem-env fallback-home copies become BEST-EFFORT once at least one store copy applied: a variant mismatch or write failure there is a per-copy non-fatal Skipped event (`gem_fallback_home_skipped`, detail names the path and reason; gated stderr twin), never a run failure. Parity edge kept: with NO bundle-store copy (the historic fallback-only layout, and every --global run) the home copy IS primary and keeps loud-fail exactly as pre-#218 apply. TDD evidence (red -> green on the rebased tip): scan/apply --json missing the warnings[] entry and the --silent leak (3 tests, in_process_gem_config_warning.rs); mismatched-home-copy exit 1 and Failed-event-on-strict-refusal (in_process_gem_fallback_home.rs), with fallback-only loud parity and both-copies-patched pinned green throughout. CLI_CONTRACT.md documents the copy classes and the warning channels. Gates: touched files rustfmt-clean; cargo clippy --workspace --all-features -D warnings clean; core --lib 2535, cli --lib 436; crawlers::ruby 50, crawler_ruby_e2e 25; gem+npm multicopy 2+2; apply_network 11, apply_invariants 4, cli_gem_variant_mismatch_policy 6, cli_apply_silent 2, e2e_gem hermetic 8 (6 shown +cache selftests), e2e_scan, cli_scan_silent, docker_e2e_gem — all green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mikola Lysenko (mikolalysenko)
added a commit
that referenced
this pull request
Aug 20, 2026
…every coexisting copy, contain config-sourced roots (#222) Post-merge audit follow-up on #218 (gem crawler flat-BUNDLE_PATH discovery), mirroring the #216 npm multi-copy precedent (0433bcb). 1. MULTI-COPY (most severe): bundler's scoped `<engine>/<abi>/gems` and flat `gems/` stores coexist under one root, each holding a REAL physical copy of the same gem@version — exactly the state #218's tests create. First-wins merging resolved the purl to ONE path, so apply patched one store and reported success while the other bundler loaded pristine (vulnerable) bytes. Fix mirrors #216: `find_all_packages_for_purls` now routes the release-variant ecosystems through an accumulating `merge_variant_copies` (reusing `push_path`, base-PURL keyed, precedence order kept) and apply's variant branch fans out per copy for gem — per-copy Applied events, `summary.applied` counts each copy, a copy matching no variant fails loudly. Rollback already carried every copy via `merge_qualified` and its per-path grouping; the new in-process suite pins both directions. PyPI/Maven deliberately keep the one-representative contract (the nuget `already_patched` double-patch regression from #216's second commit), as do all collapsing consumers (vendor/vex/setup/get/repair-vendor). scan --sync patches every copy too (it runs the real nested apply); scan's inventory stays purl-level, byte-identical to npm's crawl_all purl-dedup precedent. 2. PRECEDENCE: roots probed local-config > env > default vendor/bundle (bundler's real precedence; the old order was inverted), so first-representative consumers pick the copy bundler actually loads. 3. REGRESSION vs pre-#218: env/config roots no longer trip the `gem env` fallback early-return — only the historic project-local vendor/bundle probe keeps it. Default gems (rexml/json) live only in the DEFAULT/system gem homes, so an env-BUNDLE_PATH project gets those homes appended (deduped) again. 4. SECURITY: a config-sourced BUNDLE_PATH (committed .bundle/config = attacker-authored input, and a scan/apply WRITE-target root) must now, after ~ expansion and lexical normalization, stay contained in the project root — otherwise the root is skipped with a `gem_bundle_config_path_ignored` stderr warning naming the value. Windows rooted forms (`\evil`, `C:evil`) take the strict branch. Env-sourced BUNDLE_PATH stays trusted (user's own environment) but is normalized for dedup. `normalize_lexically` is hoisted from the composer crawler into utils::fs and shared. 5. `~` EXPANSION: a leading `~`/`~/...` in BUNDLE_PATH expands against home (bundler File.expand_path), env-injectable via the _with_env seams for hermetic tests. 6. BUNDLE_PATH__SYSTEM: `"true"` makes bundler ignore the recorded path — the config entry now parses as unset and the fallback finds the system gem homes. 7. TEST HERMETICITY: the six crawler_ruby_e2e tests that read ambient BUNDLE_PATH/BUNDLE_APP_CONFIG now route through the new `get_gem_paths_with_env` seam; a new e2e pins env-root + gem-env fallback coexistence (finding 3). 8. CLI_CONTRACT.md: the stale "gem inspects only <cwd>/vendor/bundle" claim replaced with the real root model, the containment policy, and the multi-copy behavior. TDD evidence (red -> green): dispatch-level `find_all_packages_for_purls_carries_every_gem_store_copy`, crawler `bundle_roots_probe_in_bundler_precedence_order`, and the new `in_process_gem_multicopy.rs` (real binary apply/rollback over a coexisting two-store tree) all failed on base — the flat copy stayed byte-for-byte VULNERABLE while apply reported success — and pass now. Live era-image proof (docker run --rm, socket-patch-test-gem-b1:gemx, bundler 1.17.3): the baked pre-fix binary on a coexist fixture reports status=success/applied=1 while the flat store's copy — loaded via real GEM_HOME resolution in the container — still evaluates VULNERABLE; the fixed binary reports applied=2 and both stores load FIXED. Gates: touched files rustfmt-clean; cargo clippy --workspace --all-features -D warnings clean; core --lib 2413 passed; cli --lib 430 passed; crawler_ruby_e2e 25, crawler_composer_e2e 32, crawlers_empty_paths_e2e 13; e2e_gem hermetic 8; in-process gem+npm multicopy suites; cli_gem_variant_mismatch_policy 6; docker_e2e_gem, docker_e2e_vendor_gem, docker_e2e_pypi, docker_e2e_maven all green. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Defect (D1, gem live-matrix campaign 2026-08-19 — adversarially verified 2/2 in fresh containers)
Bundler 1.x with
BUNDLE_PATHset via the environment variable installs GEM_HOME-style into the flat<BUNDLE_PATH>/gems/— no<engine>/<abi>scope segment, siblingspecifications/dir present. The ruby crawler only enumerated the scopedvendor/bundle/<engine>/<abi>/gemslayout, so on such projects:scan --jsonreported the installed gem"notInstalled": true(lockfile-only), andget <purl>returnedpartial_failuredownloaded 1 / applied 0 (package_not_installed) — the patch never landed, while the gem sat on disk atvendor/bundle/gems/activestorage-6.0.3.The
gem envfallback missed it too (the flat store is not a gem homegem envreports). Verified live against bundler 1.17.3 / ruby 3.1.7 withactivestorage@6.0.3.Root cause
RubyCrawler::get_vendor_bundle_pathshardcoded the scoped shape: it walkedvendor/bundle/<engine>/<version>/gemsonly.Bundler.bundle_pathon bundler 1 appendsruby_scopeonly for--path/local-config installs; an env-varBUNDLE_PATHis used verbatim as a GEM_HOME, producing the flat layout. bundler >= 2 appends the scope in both styles, so the gap is bundler-1-env-specific — but the flat layout is also plain GEM_HOME semantics, so it can legitimately appear under any explicit bundle root.Layout matrix
--path/bundle config set --local path<root>/<engine>/<abi>/gems(scoped)--pathremoved; config/env only)BUNDLE_PATH<root>/gems(FLAT) — was invisible(
<engine>/<abi>=Bundler.ruby_scope, e.g.ruby/3.1.0; flat roots carry a siblingspecifications/dir.)Fix
All of scan/get/apply flow through
get_gem_paths, so one fix heals all three (crates/socket-patch-core/src/crawlers/ruby_crawler.rs):<root>/<engine>/<abi>/gemswalk as before, plus flat<root>/gemsguarded on the siblingspecifications/directory every real gem home carries — a randomgems/dir is not mistaken for a gem store.BUNDLE_PATHinstall root: the env var, and theBUNDLE_PATH:entry of bundler's app config file ($BUNDLE_APP_CONFIG/config, else<cwd>/.bundle/config— reusingsetup::gem::bundler_app_config_dir, nowpub(crate)). Explicit roots are gated on the cwd holding a Bundler manifest (same "looks like a Ruby project" gate as thegem envfallback), so a machine-wideBUNDLE_PATHexport cannot pull another project's gem store into a non-Ruby scan. Relative values resolve against the project root, matchingBundler.bundle_path.gems/dirs (env var namingvendor/bundledoesn't scan twice), and skip a flat store'sgems/entry in the engine walk so a gem shipping its owngems/subdir can't surface a ghost<engine>/<version>/gemsroot.parse_bundle_config_path), matching the repo's Cargo.toml line-parsing convention; env reads live in a thin wrapper overget_vendor_bundle_paths_with_envso tests stay hermetic.Tests (test-first: RED before the fix, GREEN after)
RED run reproduced the defect exactly — flat store invisible, coexist walk surfacing the ghost
weird-1.0.0/gemsengine path, andcrawl_allfalling through to the host'sgem envhomes instead of the project store.New unit tests in
ruby_crawler.rs:get_vendor_bundle_paths_flat_bundler1_layout— flatvendor/bundle/gems+specifications/discovered (RED before fix)get_vendor_bundle_paths_ignores_bare_gems_dir— baregems/withoutspecifications/rejectedget_vendor_bundle_paths_scoped_and_flat_coexist— both layouts under one root, exactly once, ghost-engine guard (RED)crawl_all_finds_flat_bundler1_project— full local-mode pipeline on the repro shape (RED)bundle_path_env_discovers_both_layouts,bundle_path_env_relative_resolves_against_project_root,bundle_path_env_ignored_without_manifest,bundle_path_env_duplicate_root_dedupsapp_config_bundle_path_discovered,app_config_env_relocates_config,parse_bundle_config_path_contractGates:
cargo test -p socket-patch-core --lib crawlers::ruby— 40 passedcargo test -p socket-patch-core --lib— 2393 passed, 0 failedcargo test -p socket-patch-cli --test e2e_gem— 8 passed, 3 ignored (hermetic suite)cargo clippy -p socket-patch-core --lib --tests -- -D warnings— cleancargo fmt --check— clean on touched filesLive proof (fixed binary, campaign container)
Re-ran the campaign repro in
socket-patch-test-gem-b1:gemx(ruby 3.1.7, bundler 1.17.3) with the fixed binary (linux arm64 release build of this branch) bind-mounted over/usr/local/bin/socket-patch— realbundle installwith envBUNDLE_PATH, realscan/getagainst production:All three defect checks that failed 2/2 pre-fix now pass.
Campaign repro
scratchpad/gemx/lanes/agent-b1/run-bundle-path-env-layout.sh(gem live-matrix 2026-08-19, imagesocket-patch-test-gem-b1:gemx: ruby 3.1.7, bundler 1.17.3) — exits non-zero while the defect is present:DEFECT(scan)+DEFECT(get/apply)+DEFECT(disk).🤖 Generated with Claude Code
Note
Medium Risk
Changes local gem path discovery used by scan/get/apply; incorrect paths could miss or mis-target patches, but behavior is heavily unit-tested and explicit roots are gated on Bundler manifests.
Overview
Fixes gem scan/get/apply missing gems when Bundler installs into a flat
gems/tree (Bundler 1 with envBUNDLE_PATH) instead of only the scopedvendor/bundle/<engine>/<abi>/gemslayout.The Ruby crawler now probes each Bundler install root in both layouts: the existing scoped walk plus flat
<root>/gemswhen a siblingspecifications/dir exists, with deduping and a guard so a flat store’sgems/entry isn’t treated as an engine directory.Install roots are no longer just
vendor/bundle: envBUNDLE_PATHand theBUNDLE_PATH:line in Bundler app config (via sharedbundler_app_config_dir, nowpub(crate)) are honored when the cwd has a Bundler manifest; relative paths resolve against the project root. Explicit roots stay gated so a globalBUNDLE_PATHdoesn’t pollute non-Ruby scans.New helpers
resolve_bundle_pathandparse_bundle_config_pathplus a large unit-test suite cover flat layout, coexistence with scoped layout, env/config paths, and the fullcrawl_allpath.Reviewed by Cursor Bugbot for commit 698701c. Configure here.