Skip to content

fix: remove the shared temp directory in the post hook - #82

Open
Bnjoroge1 wants to merge 2 commits into
pullfrog:mainfrom
Bnjoroge1:fix/cleanup-temp-dir-upstream
Open

Bnjoroge1 wants to merge 2 commits into
pullfrog:mainfrom
Bnjoroge1:fix/cleanup-temp-dir-upstream

Conversation

@Bnjoroge1

Copy link
Copy Markdown

Problem

createTempDirectory() (utils/setup.ts:19) does:

const sharedTempDir = mkdtempSync(join(tmpdir(), "pullfrog-"));

and nothing ever removes it. entryPost.ts (post-if: always()) — the natural place — contains no rm.

This looks like an oversight rather than a decision, because the same codebase disposes every other temp dir it creates:

file line
utils/vertex.ts 73
utils/codexAuth.ts 111
mcp/xrepo.ts 151

The one that leaks is the big one — it holds the repo clone.

Why it is easy to miss

On GitHub-hosted runners the whole VM is destroyed after the job, so the leak is invisible. It only affects self-hosted runners, where it accumulates one clone per run forever.

It is worst when /tmp is a tmpfs, because then the leak is RAM, not disk. On our host (/tmp = tmpfs at 50% of RAM) each run cost ~469 MB and we reached 58 leaked clones — 12 GB, /tmp 100% full — in a single day on a 23.3 GB box:

tmpfs  12G  12G  0  100% /tmp

That left too little memory for the other workloads on the machine, and the kernel OOM-killed unrelated jobs four times.

Change

  • utils/setup.tscore.saveState("pullfrog_temp_dir", sharedTempDir). The existing process.env.PULLFROG_TEMP_DIR cannot reach the post hook: post: is a separate node process and never sees a process.env mutation from the main step. $GITHUB_STATE crosses that boundary and is already explicitly preserved by wipeRunnerLeakSurface().
  • entryPost.tscleanupTempDir() invoked via .finally().

Design notes:

  • Runs from post: rather than a finally in the main step, so it also fires on cancellation, timeout, and unhandled errors — the paths that leak most.
  • Runs after main(), because the Codex write-back reads files that may live inside the temp dir.
  • Guarded: only removes a directory whose leaf starts with pullfrog-.
  • Best-effort — a failure is warned, never thrown. The workflow is already finished.
  • The locked stdlib-only import surface from #834 is unchanged: the leaf is split by hand rather than importing node:path, so entryPost.stdlibOnly.test.ts still pins ["./utils/codexRefreshDetect.ts", "./utils/ghaCore.ts", "./utils/postApiFetch.ts", "node:fs"].

Validation

  • pnpm typecheck — clean (against upstream main, 0.1.65)
  • entryPost.stdlibOnly.test.ts — 3/3 pass (all three invariants, including the locked import surface)
  • utils/setup.test.ts — 4/4 pass

createTempDirectory() mkdtempSync's a pullfrog-XXXXXX directory under the
system temp dir and nothing ever removed it, so every run left its checkout
behind.

On GitHub-hosted runners this is invisible: the VM is destroyed with the job.
Self-hosted runners accumulate one clone per run forever, and it is worst when
/tmp is a tmpfs, because then the leak is RAM rather than disk. One host
reached 58 leaked clones -- 12 GB, /tmp 100% full -- in a single day on a
23.3 GB box, which starved it badly enough that the kernel OOM-killed the
runner VM and unrelated jobs four times.

The action already disposes its narrower temp dirs (utils/vertex.ts:73,
utils/codexAuth.ts:111, mcp/xrepo.ts:151); the shared one holding the repo
clone was the only one leaking.

Cleanup runs from post: rather than a finally in the main step so it also
fires on cancellation, timeout, and unhandled errors -- the paths that leak
most -- and after main() because the Codex write-back reads files that may
live inside the directory. The path travels through $GITHUB_STATE because
post: is a separate process and never sees a process.env mutation; that file
is explicitly preserved by wipeRunnerLeakSurface(). Best-effort: a failure
here never fails the job.

entryPost's locked stdlib-only import surface (#834) is unchanged -- the leaf
is split by hand instead of importing node:path.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No functional issues found — one inaccurate comment and some stale docs.

Reviewed changes — the full two-file diff at 31baf58, plus the $GITHUB_STATE handoff path, every writer into PULLFROG_TEMP_DIR, and the sandbox threat model around agent-controlled state.

  • Temp dir path handed to the post: hookcreateTempDirectory() now calls core.saveState("pullfrog_temp_dir", sharedTempDir) so the separate post-step process can find the directory that process.env.PULLFROG_TEMP_DIR cannot reach.
  • Recursive cleanup in entryPost.ts — a new cleanupTempDir() runs via .finally() after main(), guarded on a pullfrog- leaf, warning-only on failure, using rmSync added to the existing node:fs import so the locked stdlib-only surface is unchanged.

The mechanism holds up. runCli.ts passes { ...process.env } to the CLI child, so GITHUB_STATE reaches utils/setup.ts and the file-based saveState path is taken (never the deprecated stdout fallback); wipeRunnerLeakSurface() explicitly preserves that file; and getState reads it back as STATE_pullfrog_temp_dir — the identical, already-in-production mechanism as codex_writeback. Nothing written into the temp dir outlives the main step: utils/install.ts's "cache" is per-run because the parent is mkdtempSync'd fresh, background shell processes are killed by killBackgroundProcesses before MCP teardown, and the path is never exposed as an action output. pnpm typecheck is clean and entryPost.stdlibOnly.test.ts (3) + utils/setup.test.ts (4) pass on this branch.

ℹ️ The post hook now has two jobs, but its own documentation still says one

entryPost.ts and action.yml both state in prose that the Codex write-back is the only thing the post: step does. This PR makes that false, and neither line is in a diff hunk so it cannot be flagged inline. These are the two places a future reader goes to learn what the hook is for, so leaving them stale is how the next person concludes the cleanup is dead code.

Technical details
# Stale "only job" / "only consumer" claims about the `post:` hook

## Affected sites
- `entryPost.ts:17` — file header: "Today's only job: detect a Codex auth refresh by diffing the on-disk auth.json ...". The hook now also removes the shared temp directory.
- `action.yml:54-56` — "Always-run post step persists best-effort state ... Today's only consumer: Codex auth.json refresh write-back." Temp-dir cleanup is now a second consumer, and notably it is NOT "persisting state" — it is the first consumer that exists to release a resource.

## Required outcome
- Both comments enumerate the hook's responsibilities accurately after this change, so the `post-if: always()` contract reads as covering cleanup as well as credential persistence.

ℹ️ Nitpicks

  • runCli.ts:174 still does mkdtempSync(join(tmpdir(), "pullfrog-bootstrap-")) with no counterpart, so the self-hosted /tmp this PR is fixing keeps accumulating one directory per run. It stays empty (npx --yes installs into ~/.npm/_npx), so the bytes are noise next to the 469 MB clone — but it is the one case the PR description's "the codebase disposes every other temp dir it creates" framing doesn't actually cover, and it would match the new pullfrog- guard if it were ever wired up.

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using Claude Opus (free via Pullfrog for OSS) | 𝕏

Comment thread entryPost.ts Outdated
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant