Skip to content

fix(etcd): clean up orphaned data PVCs and TLS secrets on tenant module delete (#3090) - #3170

Merged
scooby87 merged 3 commits into
mainfrom
fix/etcd-pvc-secrets-orphaned
Jul 2, 2026
Merged

fix(etcd): clean up orphaned data PVCs and TLS secrets on tenant module delete (#3090)#3170
scooby87 merged 3 commits into
mainfrom
fix/etcd-pvc-secrets-orphaned

Conversation

@scooby87

@scooby87 scooby87 commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

What this PR does

When the etcd tenant module is disabled (etcd: false), the etcd HelmRelease is pruned but two classes of resources leak into the tenant namespace (reported in #3090):

  • Data PVCs — the etcd-operator StatefulSet volumeClaimTemplate PVCs data-etcd-<ordinal> are never garbage-collected on EtcdCluster deletion.
  • TLS secrets — the five etcd-*-tls secrets survive the uninstall:
    • etcd-ca-tls / etcd-client-tls are created by this chart as pre-install hooks with helm.sh/resource-policy: keep, so Helm deliberately keeps them;
    • etcd-peer-ca-tls / etcd-server-tls / etcd-peer-tls are populated by cert-manager from Certificate resources and carry no owner reference back to the chart.

This adds a post-delete cleanup hook (packages/extra/etcd/templates/hooks/cleanup.yaml), modeled on the accepted seaweedfs (#3092) and monitoring (#3091) sibling fixes, that:

  • deletes the data PVCs by name, one per replica (data-etcd-0 … data-etcd-<replicas-1>);
  • deletes the five etcd-*-tls secrets by name.

check-release-name pins the release name to etcd, so every target name is deterministic and no sibling workload shares the data-etcd-* prefix in a tenant namespace — deletion by name is safe. cert-manager TLS secrets carry only the cluster-wide controller.cert-manager.io/fao label, so the RBAC grants delete by explicit resourceNames rather than an unsafe label sweep.

The cleanup Job is hardened: non-root, read-only root fs, dropped capabilities, seccompProfile: RuntimeDefault, resource requests/limits, activeDeadlineSeconds. Deletes use --ignore-not-found and failures are logged but never block teardown.

A Helm unit-test suite (tests/cleanup_test.yaml) asserts the hook metadata/RBAC/hardening and that the rendered commands target exactly the reported PVCs and secrets, including scaling the PVC list to the configured replica count.

Note

The stuck HelmRelease finalizer noted in #3090 is the platform-wide sharded-helm-controller side-bug (tracked separately); it affects every chartRef-based tenant module identically, not just etcd. This PR fixes the PVC and secret orphans the etcd chart itself owns, matching the scope of the merged seaweedfs/monitoring fixes.

Warning

Disabling the etcd module now permanently deletes its data PVCs and the etcd data they hold. This is intentional and fixes the storage leak. Back up before disabling if the data matters.

Fixes #3090.

Release note

fix(etcd): garbage-collect the orphaned etcd data PVCs (data-etcd-*) and the five etcd-*-tls secrets left behind when a tenant's etcd module is disabled. WARNING: disabling etcd now permanently removes its data PVCs and the data they hold.

Summary by CodeRabbit

  • New Features
    • Added a Helm post-delete cleanup hook for etcd that removes per-replica etcd PVCs and the related TLS secrets when a release is deleted.
    • Cleanup permissions and targets now scale with replica count, including safe handling when replicas are set to zero.
  • Tests
    • Added Helm unit tests validating hook annotations/order, scoped RBAC, secure job hardening, and the exact deletion behavior for PVCs/secrets (including higher-replica scenarios).
  • Chores
    • Marked the test make target as phony for the etcd package.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a resource leakage issue where etcd data PVCs and TLS secrets persist in the tenant namespace after the etcd module is disabled. By implementing a hardened post-delete cleanup Job, the system now ensures these orphaned resources are garbage-collected, preventing storage and secret accumulation. Note that this change results in the permanent deletion of etcd data upon disabling the module.

Highlights

  • Orphaned Resource Cleanup: Introduced a post-delete Helm hook to automatically remove etcd data PVCs and TLS secrets when the etcd module is disabled.
  • Security Hardening: The cleanup Job is configured with non-root execution, read-only root filesystem, dropped capabilities, and restricted resource limits.
  • Testing: Added comprehensive Helm unit tests to verify RBAC permissions, Job configuration, and correct command generation for PVC/secret deletion.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment Gemini (@gemini-code-assist) Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@github-actions github-actions Bot added size/L This PR changes 100-499 lines, ignoring generated files area/database Issues or PRs related to managed databases (postgres, mariadb, redis, etcd, kafka, clickhouse) kind/bug Categorizes issue or PR as related to a bug labels Jul 1, 2026
@dosubot dosubot Bot added the area/extra Issues or PRs related to tenant-specific modules (packages/extra/) label Jul 1, 2026
@coderabbitai

coderabbitai Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 7997d879-0889-4321-a0b3-590e72e6a618

📥 Commits

Reviewing files that changed from the base of the PR and between 4a06df9 and 9472bb3.

📒 Files selected for processing (2)
  • packages/extra/etcd/templates/hooks/cleanup.yaml
  • packages/extra/etcd/tests/cleanup_test.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/extra/etcd/tests/cleanup_test.yaml

📝 Walkthrough

Walkthrough

This PR adds a post-delete etcd cleanup hook that deletes orphaned PVCs and TLS secrets, with scoped RBAC, a hardened Job, helm-unittest coverage, and a phony Makefile test target.

Changes

Cleanup hook implementation and tests

Layer / File(s) Summary
Cleanup hook RBAC
packages/extra/etcd/templates/hooks/cleanup.yaml
Adds the cleanup ServiceAccount, Role, and RoleBinding with post-delete metadata and scoped permissions for the targeted PVCs and TLS secrets.
Cleanup Job execution
packages/extra/etcd/templates/hooks/cleanup.yaml
Adds a post-delete Job that runs kubectl with hardened settings to delete the per-replica PVCs and named TLS secrets, ignoring missing resources.
Tests and Makefile support
packages/extra/etcd/tests/cleanup_test.yaml, packages/extra/etcd/Makefile
Adds helm-unittest coverage for hook metadata, RBAC, Job spec, and delete commands, plus .PHONY: test in the Makefile.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Helm
  participant ServiceAccount
  participant Role
  participant RoleBinding
  participant CleanupJob
  participant KubeAPI

  Helm->>ServiceAccount: render post-delete hook identity
  Helm->>Role: render scoped delete permissions
  Helm->>RoleBinding: bind Role to ServiceAccount
  Helm->>CleanupJob: trigger post-delete hook
  CleanupJob->>KubeAPI: delete data-etcd-<n> PVCs
  CleanupJob->>KubeAPI: delete named TLS secrets
  KubeAPI-->>CleanupJob: deletion result
Loading

Suggested labels: kind/breaking-change, area/storage

Suggested reviewers: kvaps, lllamnyp, androndo, IvanHunters, sircthulhu, myasnikovdaniil, lexfrei

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: cleaning up orphaned etcd PVCs and TLS secrets when the tenant module is deleted.
Linked Issues check ✅ Passed The PR implements the requested post-delete cleanup for orphaned PVCs and TLS secrets and adds tests covering the hook behavior.
Out of Scope Changes check ✅ Passed The changes stay focused on etcd cleanup and test support; the Makefile tweak only marks the test target phony.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/etcd-pvc-secrets-orphaned

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/extra/etcd/tests/cleanup_test.yaml (1)

46-48: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Assert the Role and RoleBinding are actual post-delete hooks.

These tests only check hook weight. Add assertions for metadata.annotations["helm.sh/hook"] and metadata.annotations["helm.sh/hook-delete-policy"] so a regression cannot render the RBAC as normal release resources.

Suggested test coverage addition
       - equal:
+          path: metadata.annotations["helm.sh/hook"]
+          value: post-delete
+      - equal:
           path: metadata.annotations["helm.sh/hook-weight"]
           value: "5"
+      - equal:
+          path: metadata.annotations["helm.sh/hook-delete-policy"]
+          value: before-hook-creation,hook-succeeded

Apply the same assertion block to both the Role and RoleBinding tests.

Also applies to: 73-75

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/extra/etcd/tests/cleanup_test.yaml` around lines 46 - 48, The Role
and RoleBinding cleanup tests only verify hook weight, so extend both assertion
blocks to also check metadata.annotations["helm.sh/hook"] and
metadata.annotations["helm.sh/hook-delete-policy"]. Update the existing Role and
RoleBinding test cases in cleanup_test.yaml so they explicitly confirm these
RBAC objects are post-delete hooks and not normal release resources.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/extra/etcd/templates/hooks/cleanup.yaml`:
- Around line 24-26: The PVC delete permissions in the cleanup Role are too
broad because the `persistentvolumeclaims` rule allows namespace-wide access.
Tighten the RBAC in the cleanup template by adding generated `resourceNames` for
the deterministic `data-etcd-*` PVC names and removing `list` if the cleanup Job
only targets known names; keep the rule name-based and aligned with the existing
least-privilege pattern used for secrets.

---

Nitpick comments:
In `@packages/extra/etcd/tests/cleanup_test.yaml`:
- Around line 46-48: The Role and RoleBinding cleanup tests only verify hook
weight, so extend both assertion blocks to also check
metadata.annotations["helm.sh/hook"] and
metadata.annotations["helm.sh/hook-delete-policy"]. Update the existing Role and
RoleBinding test cases in cleanup_test.yaml so they explicitly confirm these
RBAC objects are post-delete hooks and not normal release resources.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e38358a6-013f-4344-b4c6-aab85eb481bc

📥 Commits

Reviewing files that changed from the base of the PR and between 4783add and 1cea32c.

📒 Files selected for processing (3)
  • packages/extra/etcd/Makefile
  • packages/extra/etcd/templates/hooks/cleanup.yaml
  • packages/extra/etcd/tests/cleanup_test.yaml

Comment thread packages/extra/etcd/templates/hooks/cleanup.yaml Outdated
scooby87 added a commit that referenced this pull request Jul 2, 2026
…ook metadata

Address CodeRabbit review on #3170:
- Tighten the cleanup Role's PVC rule from a namespace-wide get/list/delete to
  get/delete by explicit resourceNames (data-etcd-<ordinal>, one per replica),
  mirroring the least-privilege pattern already used for the TLS secrets. The
  Job only ever deletes those deterministic names, so list is unnecessary and a
  regression can no longer reach a sibling workload's PVC.
- Extend the Role and RoleBinding unit tests to assert helm.sh/hook and
  helm.sh/hook-delete-policy (not just the weight), so the RBAC cannot regress
  into normal release resources, and assert the PVC resourceNames.

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>
scooby87 and others added 2 commits July 2, 2026 15:58
…le delete (#3090)

When the etcd tenant module is disabled (etcd: false), the etcd-operator
StatefulSet data PVCs (data-etcd-<ordinal>) are never garbage-collected, and
the five etcd-*-tls secrets are left orphaned in the tenant namespace:
etcd-ca-tls / etcd-client-tls are pre-install hooks this chart marks with
helm.sh/resource-policy: keep, and etcd-peer-ca-tls / etcd-server-tls /
etcd-peer-tls are cert-manager-issued secrets with no owner reference back to
the chart. None are reclaimed by the Helm uninstall.

Add a post-delete cleanup Job (modeled on the seaweedfs/monitoring sibling
fixes) that deletes the data PVCs by name (one per replica) and the five TLS
secrets by name. check-release-name pins the release to "etcd", so every
target name is deterministic and no sibling workload shares the data-etcd-*
prefix. The Job runs non-root with a read-only root fs, dropped capabilities
and seccompProfile RuntimeDefault; its RBAC is scoped to PVC get/list/delete
and the five secrets by resourceNames. Deletes use --ignore-not-found and
failures are logged but never block teardown.

The stuck-HelmRelease finalizer noted in the issue is the platform-wide
sharded-helm-controller side-bug tracked separately; this change addresses the
PVC and secret orphans the etcd chart owns.

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>
…ook metadata

Address CodeRabbit review on #3170:
- Tighten the cleanup Role's PVC rule from a namespace-wide get/list/delete to
  get/delete by explicit resourceNames (data-etcd-<ordinal>, one per replica),
  mirroring the least-privilege pattern already used for the TLS secrets. The
  Job only ever deletes those deterministic names, so list is unnecessary and a
  regression can no longer reach a sibling workload's PVC.
- Extend the Role and RoleBinding unit tests to assert helm.sh/hook and
  helm.sh/hook-delete-policy (not just the weight), so the RBAC cannot regress
  into normal release resources, and assert the PVC resourceNames.

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>
@scooby87
scooby87 force-pushed the fix/etcd-pvc-secrets-orphaned branch from 8fb4bfc to 4a06df9 Compare July 2, 2026 12:58
@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Caution

Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted.

Error details
{}

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/extra/etcd/templates/hooks/cleanup.yaml`:
- Around line 27-33: The cleanup hook template is allowing an explicit replicas:
0 to render an empty resourceNames list in the cleanup job. Update the hook
logic in the cleanup.yaml template around the .Values.replicas usage so it
clamps the count to at least 1 or emits a safe sentinel name, and keep the fix
localized to the resourceNames generation for the PVC cleanup path.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: baba44d9-eb48-4658-9d90-5652929eaec5

📥 Commits

Reviewing files that changed from the base of the PR and between 8fb4bfc and 4a06df9.

📒 Files selected for processing (3)
  • packages/extra/etcd/Makefile
  • packages/extra/etcd/templates/hooks/cleanup.yaml
  • packages/extra/etcd/tests/cleanup_test.yaml
✅ Files skipped from review due to trivial changes (1)
  • packages/extra/etcd/Makefile
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/extra/etcd/tests/cleanup_test.yaml

Comment thread packages/extra/etcd/templates/hooks/cleanup.yaml
… RBAC

Address CodeRabbit review on #3170: the PVC cleanup Role generates its
resourceNames from .Values.replicas. values.schema.json sets no minimum, so an
explicit replicas: 0 rendered an empty resourceNames list — and an empty
resourceNames in an RBAC rule matches every PVC in the namespace (a wildcard),
silently undoing the least-privilege scoping the previous commit added.

Clamp the count to max(1, replicas) via a shared $replicaCount used by both the
Role resourceNames range and the delete command, so at least data-etcd-0 is
always listed (it exists for any running cluster and the delete uses
--ignore-not-found). Add a regression test asserting replicas: 0 still yields a
name-scoped Role and a single-name delete rather than a namespace-wide grant.

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>

@IvanHunters IvanHunters left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: LGTM with non-blocking notes

Correct, narrowly-scoped post-delete cleanup hook, functionally identical to the merged seaweedfs sibling (#3092). RBAC is scoped by name, cross-namespace/cross-tenant match is impossible. helm unittest passes (26 tests).

Verified

  • All deletes are -n {{ .Release.Namespace }} with a namespaced Role, so cross-namespace match is impossible. PVCs are deleted by explicit resourceNames: data-etcd-<i>; the five secrets by literal name, matching exactly the secretName values in etcd-cluster.yaml.
  • The max 1 (int .Values.replicas) clamp prevents an empty resourceNames (an empty list would act as a wildcard over all PVCs). Covered by the replicas: 0 test.
  • The pre-install keep-policy placeholder secrets (etcd-ca-tls, etcd-client-tls in datastore.yaml) are why a normal uninstall leaves them behind, which is why the explicit delete is needed.
  • The Job carries policy.cozystack.io/allow-to-apiserver: "true", required for egress to the API server under the tenant default-deny policy. PSS-restricted securityContext is complete.

Non-blocking notes

  • The PR body says the fix is modeled on merged seaweedfs and monitoring siblings; the monitoring cleanup hook is not merged yet (it is PR #3094). Only seaweedfs is a merged precedent.
  • Pre-existing, not touched by this PR: tests/no-post-upgrade-hook_test.yaml lists templates/hook/job.yaml and templates/version.yaml, which do not exist in the tree, so those count: 0 asserts pass trivially. Worth cleaning up in a follow-up.
  • Delete commands use || echo WARNING without set -e, so the Job never fails a teardown on a failed delete. This is a deliberate "never block teardown" choice and matches seaweedfs; it differs from clickhouse (set -eu; exit 1).
  • Consistency: this PR introduces permanent data-PVC deletion on module disable but is not marked as a breaking change, while the equivalent monitoring PR (#3094) carries the ! breaking-change marker. Consider aligning the marker for symmetry.

Upgrade impact: the hook fires only on etcd-module uninstall (etcd: false), so a normal upgrade that keeps the module changes nothing. The intended breaking change (disabling the etcd module now permanently deletes its data PVCs, per #3090) is documented in the PR body WARNING. Fresh install: hook is inactive on install/upgrade.

Family consistency

  • Across the sibling PRs the PVC-deletion strategy is not uniform: etcd deletes by explicit name, monitoring (#3094) by label, seaweedfs by instance label. Each is safe on its own; worth aligning eventually.

@scooby87
scooby87 merged commit a162681 into main Jul 2, 2026
16 checks passed
@scooby87
scooby87 deleted the fix/etcd-pvc-secrets-orphaned branch July 2, 2026 15:21
scooby87 added a commit that referenced this pull request Jul 2, 2026
…nd route it through cozy-lib.images-registry (#3171)

## What this PR does

Fixes #3088.

The post-delete cleanup Jobs in the **harbor** and **mariadb** charts
hardcoded the cleanup image as `docker.io/clastix/kubectl:v1.32` — a
moving Docker Hub tag. This:

- **broke reproducibility / was a supply-chain risk** — `v1.32` is
mutable and can be re-pushed to a different image;
- **could not be satisfied on air-gapped / mirrored clusters** — the
hardcoded `docker.io` bypassed the cluster images registry, and it
failed at the worst moment: during **uninstall**, when the `post-delete`
hook runs and a stuck Job stalls release teardown.

### Changes

- **New canonical helper `cozy-lib.image`**
(`packages/library/cozy-lib/templates/_cozyconfig.tpl`). Given a
registry-relative image path and the global context, it prefixes
`cozy-lib.images-registry` when set (`<registry>/<image>`) and returns
the image unchanged when empty — **with no leading `/`**, so standard
installs still resolve from the default registry. This establishes the
previously-undefined "how to reference a cluster image" pattern
(`images-registry` was defined but never consumed).
- **Digest-pinned image** in both cleanup hooks:
`clastix/kubectl:v1.32@sha256:b9ef7d8dbe65bcc81a46c09b8dc7543103055021c4f43287bf59e92a8f4fe05c`
(the immutable multi-arch OCI index for the `v1.32` tag; the same digest
already pinned for `postgres-operator`'s webhook hook).
- **Renovate keeps the pin fresh.** The repo disables the helm-values
manager (`enabledManagers`), and the ref is assembled at render time by
the helper rather than being a Dockerfile / go.mod dependency, so the
built-in managers can't see it. A `custom.regex` manager over the hook
templates tracks `clastix/kubectl` by tag+digest — mirroring the
existing `extra/etcd` manager — so the supply-chain pin doesn't silently
age.
- **Hardened `cozy-lib.images-registry`** against a nil
`.Values._cluster` so the helper never crashes the render — relevant
precisely because this hook renders during teardown.
- **Helm unit tests** in both charts assert the rendered image for three
cases: empty registry (default), registry set, and the no-leading-slash
edge case. Assertions match by **pattern** (routed + digest-pinned + no
leading slash) rather than an exact digest, so a Renovate bump keeps
them green.

### Verification

- `helm unittest` passes for both charts (harbor: 16, mariadb: 10).
- Real `helm template` renders confirmed for both charts:
  - empty registry → `clastix/kubectl:v1.32@sha256:b9ef…`
- `images-registry=registry.internal:5000` →
`registry.internal:5000/clastix/kubectl:v1.32@sha256:b9ef…`
- The Renovate `matchStrings` regex verified to capture
`currentValue=v1.32` and the digest from both hook templates.

### Note on the wider footprint

The same `docker.io/clastix/kubectl:v1.32` hardcode is already merged on
`main` in several other cleanup hooks (bucket, qdrant, gateway,
seaweedfs, kubernetes, tenant, dashboard, keycloak-configure) and
appears in the open PRs #3170 (etcd), #3094 (monitoring), #3072
(clickhouse). This PR fixes harbor + mariadb per the issue and
establishes the `cozy-lib.image` + Renovate pattern; migrating the
remaining hooks is a follow-up.

### Release note

```release-note
fix(cozy-lib,harbor,mariadb): digest-pin the harbor and mariadb post-delete cleanup-hook kubectl image and route it through the cluster images registry (`cozy-lib.images-registry`) via the new `cozy-lib.image` helper, so mirrored / air-gapped installs can resolve it and uninstall no longer depends on a moving Docker Hub tag; a custom Renovate manager keeps the digest fresh
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Updated post-delete cleanup hook Jobs to use digest-pinned container
images for more secure, repeatable execution.
* Improved image reference rendering so cleanup hooks work correctly
with or without an image registry configured, avoiding invalid
leading/trailing slash issues.
* **Tests**
* Extended cleanup hook rendering tests to validate digest pinning and
correct registry prefix behavior, including edge cases.
* **Chores**
* Enhanced automated dependency detection to treat digest-pinned hook
image references as stable.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/database Issues or PRs related to managed databases (postgres, mariadb, redis, etcd, kafka, clickhouse) area/extra Issues or PRs related to tenant-specific modules (packages/extra/) kind/bug Categorizes issue or PR as related to a bug size/L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

etcd (tenant module): HelmRelease stuck Terminating + orphaned PVCs/TLS secrets after the module is disabled

2 participants