feat(platform): add deletion-protection guardrail via ValidatingAdmissionPolicy - #2650
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds label-based deletion protection: resources are stamped with platform.cozystack.io/no-delete=true, a ValidatingAdmissionPolicy + Binding deny DELETE for labeled objects, Helm templates add the label to critical resources, a migration backfills existing resources, and tests verify label injection and preservation. ChangesDeletion Protection Implementation
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request implements a robust deletion-protection mechanism for critical Cozystack platform resources. By leveraging Kubernetes ValidatingAdmissionPolicy, the system can now natively block DELETE requests on sensitive objects identified by a specific label. This approach provides a lightweight, infrastructure-free alternative to traditional admission webhooks, ensuring high availability and stability for core platform services. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request implements a deletion protection mechanism for critical system resources by introducing a ValidatingAdmissionPolicy that blocks the deletion of objects labeled with platform.cozystack.io/no-delete: "true". The label is applied across various components, including CRDs, namespaces, and cluster issuers. Review feedback recommends wrapping the policy in a capability check to maintain compatibility with Kubernetes versions prior to 1.30 and suggests applying the protection label to the guardrail resources themselves to prevent their accidental deletion.
| apiVersion: admissionregistration.k8s.io/v1 | ||
| kind: ValidatingAdmissionPolicy | ||
| metadata: | ||
| name: cozystack-no-delete-guardrail | ||
| spec: | ||
| failurePolicy: Fail | ||
| matchConstraints: | ||
| resourceRules: | ||
| - apiGroups: ["*"] | ||
| apiVersions: ["*"] | ||
| operations: ["DELETE"] | ||
| resources: ["*/*"] | ||
| scope: "*" | ||
| validations: | ||
| - expression: "false" | ||
| message: >- | ||
| Deletion blocked: object carries platform.cozystack.io/no-delete=true. | ||
| To bypass, first remove the label: | ||
| kubectl label <kind> <name> platform.cozystack.io/no-delete- | ||
| --- | ||
| apiVersion: admissionregistration.k8s.io/v1 | ||
| kind: ValidatingAdmissionPolicyBinding | ||
| metadata: | ||
| name: cozystack-no-delete-guardrail | ||
| spec: | ||
| policyName: cozystack-no-delete-guardrail | ||
| validationActions: [Deny] | ||
| matchResources: | ||
| objectSelector: | ||
| matchLabels: | ||
| platform.cozystack.io/no-delete: "true" |
There was a problem hiding this comment.
The ValidatingAdmissionPolicy API was introduced as GA in Kubernetes 1.30. To prevent installation or upgrade failures on clusters running older versions of Kubernetes, it is recommended to wrap these resources in a capability check. This ensures the platform chart remains backward compatible and only deploys the guardrail when the API is available.
{{- if .Capabilities.APIVersions.Has "admissionregistration.k8s.io/v1" }}
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
name: cozystack-no-delete-guardrail
spec:
failurePolicy: Fail
matchConstraints:
resourceRules:
- apiGroups: ["*"]
apiVersions: ["*"]
operations: ["DELETE"]
resources: ["*/*"]
scope: "*"
validations:
- expression: "false"
message: >-
Deletion blocked: object carries platform.cozystack.io/no-delete=true.
To bypass, first remove the label:
kubectl label <kind> <name> platform.cozystack.io/no-delete-
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
name: cozystack-no-delete-guardrail
spec:
policyName: cozystack-no-delete-guardrail
validationActions: [Deny]
matchResources:
objectSelector:
matchLabels:
platform.cozystack.io/no-delete: "true"
{{- end }}There was a problem hiding this comment.
Wrapped the policy and binding in a Helm capability gate so the template renders to nothing on clusters that pre-date 1.30, per 54903bc. Used Capabilities.APIVersions.Has "admissionregistration.k8s.io/v1/ValidatingAdmissionPolicy" rather than the bare admissionregistration.k8s.io/v1 — the group/version has existed since 1.16 for ValidatingWebhookConfiguration, so only the Kind-scoped check actually gates on 1.30+.
| apiVersion: admissionregistration.k8s.io/v1 | ||
| kind: ValidatingAdmissionPolicy | ||
| metadata: | ||
| name: cozystack-no-delete-guardrail |
There was a problem hiding this comment.
Consider adding the platform.cozystack.io/no-delete: "true" label to the ValidatingAdmissionPolicy and ValidatingAdmissionPolicyBinding resources themselves. This would prevent accidental deletion of the guardrail itself, requiring an explicit label removal to disable the protection mechanism.
name: cozystack-no-delete-guardrail
labels:
platform.cozystack.io/no-delete: "true"There was a problem hiding this comment.
Good catch — labeled both the ValidatingAdmissionPolicy and the ValidatingAdmissionPolicyBinding with platform.cozystack.io/no-delete: "true" in c8ff3cc. Direct deletion of either is now denied by the policy itself; the standard label-removal bypass remains the path for intentional removal.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/core/platform/templates/cozystack-version.yaml`:
- Around line 9-10: Existing cozystack-version ConfigMaps aren’t backfilled
because the current template only renders when lookup() returns nothing; add an
upgrade-time render/patch path that runs when
lookup("v1","ConfigMap","cozystack-version") returns a result and applies the
label platform.cozystack.io/no-delete: "true" to the existing object.
Concretely, add a second manifest conditional (the inverse of the current lookup
check) that emits a patch/ConfigMap (or Kubernetes strategic-merge patch)
targeting metadata.name: cozystack-version and sets
metadata.labels.platform.cozystack.io/no-delete = "true" so upgrades will
backfill the label for existing resources.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: cc980c48-4881-40b6-b055-8a55fdb28e81
📒 Files selected for processing (9)
internal/crdinstall/install.gopackages/core/installer/templates/cozy-system-labels.yamlpackages/core/installer/templates/cozy-system-namespace.yamlpackages/core/platform/templates/cozystack-version.yamlpackages/core/platform/templates/deletion-protection.yamlpackages/core/platform/templates/repository.yamlpackages/system/cert-manager-issuers/templates/cluster-issuers.yamlpackages/system/cozystack-basics/templates/tenant-root.yamlpackages/system/linstor/templates/cluster.yaml
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/core/platform/images/migrations/migrations/40`:
- Line 1: The script uses bash-specific "set -o pipefail" but currently has a
POSIX shebang (#!/bin/sh); update the shebang to a bash interpreter (e.g.
#!/usr/bin/env bash or #!/bin/bash) so "set -o pipefail" works and the pipeline
around the kubectl create commands (the pipeline invoked near lines with kubectl
create at lines ~30-32) correctly fails on errors; ensure the script header is
changed in the migration file (the current first line with "#!/bin/sh") to the
bash shebang so functions/commands relying on bash semantics run reliably.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 0c765201-e4a6-496c-90e9-29fd29fd384a
📒 Files selected for processing (3)
packages/core/platform/images/migrations/migrations/40packages/core/platform/templates/deletion-protection.yamlpackages/core/platform/values.yaml
| @@ -0,0 +1,32 @@ | |||
| #!/bin/sh | |||
There was a problem hiding this comment.
Fix shebang/pipefail mismatch to prevent migration failure.
The script uses #!/bin/sh but line 15 invokes set -o pipefail, which is a bash-specific feature not supported by POSIX-compliant shells like dash (default /bin/sh on Debian/Ubuntu) or some busybox ash variants (Alpine). This will cause the script to fail immediately with "set: pipefail: bad option" on many common container base images, blocking the migration from running.
The -o pipefail option is necessary here to ensure that if kubectl create fails in the pipeline at lines 30-32, the overall migration fails rather than silently continuing with partial success.
🐛 Proposed fix: change shebang to bash
-#!/bin/sh
+#!/bin/bash
# Migration 40 --> 41Also applies to: 15-15
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/core/platform/images/migrations/migrations/40` at line 1, The script
uses bash-specific "set -o pipefail" but currently has a POSIX shebang
(#!/bin/sh); update the shebang to a bash interpreter (e.g. #!/usr/bin/env bash
or #!/bin/bash) so "set -o pipefail" works and the pipeline around the kubectl
create commands (the pipeline invoked near lines with kubectl create at lines
~30-32) correctly fails on errors; ensure the script header is changed in the
migration file (the current first line with "#!/bin/sh") to the bash shebang so
functions/commands relying on bash semantics run reliably.
There was a problem hiding this comment.
Verified against the actual runtime: the migrations image is Alpine 3.22 (packages/core/platform/images/migrations/Dockerfile), where /bin/sh is BusyBox ash, which has supported set -o pipefail since BusyBox 1.30 (2018) — there's no "bad option" error in this image. Migration 38 ships with the same #!/bin/sh + set -euo pipefail pattern and runs cleanly in production, as does run-migrations.sh. Keeping #!/bin/sh here for consistency with the existing shipping migrations rather than introducing a third shebang convention.
There was a problem hiding this comment.
@myasnikovdaniil, thanks for the thorough runtime verification — that's exactly the right level of diligence. BusyBox ash on Alpine 3.22 with pipefail support confirmed, and consistency with migration 38 and run-migrations.sh is a solid reason to keep #!/bin/sh. I'll note this for future reviews.
🐇 (✿◠‿◠)
✏️ Learnings added
Learnt from: myasnikovdaniil
Repo: cozystack/cozystack PR: 2650
File: packages/core/platform/images/migrations/migrations/40:1-1
Timestamp: 2026-05-15T05:58:57.959Z
Learning: In the `cozystack` project, migration scripts under `packages/core/platform/images/migrations/migrations/` use `#!/bin/sh` with `set -euo pipefail`. This is intentional and valid: the migrations container image is Alpine 3.22 where `/bin/sh` is BusyBox ash, which has supported `set -o pipefail` since BusyBox 1.30 (2018). Do not flag this combination as a shebang/pipefail mismatch — it is the established convention across all migrations (e.g., migration 38) and `run-migrations.sh`.
Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.
There was a problem hiding this comment.
Heads-up — this file was renumbered to migrations/42 after a merge from main, because main now ships its own migrations/40 (drops dashboard CRDs) and 41 (renames ephemeralStorage → diskSize). Same content, same Alpine 3.22 / BusyBox ash runtime, so the set -euo pipefail analysis you acknowledged above still holds for the relocated script. The chart's migrations.targetVersion is now 43 so the runner replays 40 → 41 → 42 in order on upgraded clusters.
fe3ecda to
cc107fe
Compare
Timofei Larkin (lllamnyp)
left a comment
There was a problem hiding this comment.
NOT LGTM
Reviewing against main. 4 commits on top of e4f9a18.
Scope: adds a ValidatingAdmissionPolicy (+Binding) that blocks DELETE on objects labeled platform.cozystack.io/no-delete=true, stamps the label on a fixed set of platform objects (CRDs, namespaces, ConfigMap, Repository, ClusterIssuers, LinstorCluster, tenant-root HelmRelease, the VAP/Binding themselves), bumps targetVersion 40 → 41, and adds migration 40 to backfill the label on the pre-existing cozystack-version ConfigMap.
The architecture is sound (in-process CEL, objectSelector-scoped binding, self-protection via labeling the VAP/Binding, capability gate for 1.30+, migration backfill for the one resource helm can't re-render). The blockers below are about untested behavior on the Go side, a fragile one-shot pattern in the migration script, and missing test coverage for every new behavior the PR introduces.
Blockers
1. internal/crdinstall/install.go — label-stamping has no test, and the label-merge contract is unasserted
internal/crdinstall/install.go:92-101:
// Stamp the deletion-protection label so the platform VAP guards every
// Cozystack CRD against accidental kubectl delete.
for _, obj := range objects {
labels := obj.GetLabels()
if labels == nil {
labels = make(map[string]string)
}
labels["platform.cozystack.io/no-delete"] = "true"
obj.SetLabels(labels)
}The Patch call below uses client.Apply (SSA) with FieldManager: "cozystack-operator" and Force: true. SSA semantics: every field set by this field manager is owned by it; fields ABSENT from this apply that were previously owned by the same field manager get removed from the object.
The code is fine in itself — obj.GetLabels() preserves whatever the manifest YAML carried, and the new key is merged in. But internal/crdinstall/install_test.go does not assert the label is actually present on the applied object. TestInstall_appliesAllCRDs only checks that Install returns no error; it does not read the applied CRD back from the fake client and inspect labels. If a future refactor drops the loop, the test stays green and the protection is silently gone.
Required fix:
- Extend
TestInstall_appliesAllCRDs(or add a new test) to read the applied CRDs back from the fake client afterInstallreturns and assertlabels["platform.cozystack.io/no-delete"] == "true"on each. - Add a second test variant where the input manifest YAML already carries other labels (e.g.
app.kubernetes.io/managed-by: foo) and assert both labels survive — exercises theGetLabels() == nilbranch AND the merge path.
Test that fails without the fix: a unit test that builds Install with a manifest YAML carrying app.kubernetes.io/managed-by: foo, runs Install against a fake client, then fakeClient.Get for the CRD and asserts both app.kubernetes.io/managed-by == foo AND platform.cozystack.io/no-delete == true.
2. packages/core/platform/images/migrations/migrations/40 — no test, and the applied manifest does not carry the label
packages/core/platform/images/migrations/migrations/40:26-32:
kubectl label --namespace "$NAMESPACE" --overwrite configmap cozystack-version \
platform.cozystack.io/no-delete=true
echo "Labeled configmap $NAMESPACE/cozystack-version"
kubectl create configmap --namespace "$NAMESPACE" cozystack-version \
--from-literal=version=41 --dry-run=client --output yaml \
| kubectl apply --filename -The script first stamps the label imperatively, then re-applies the ConfigMap to bump data.version=41. The applied YAML carries no labels — the script relies on kubectl apply's 3-way strategic merge to leave the imperatively-set label alone. This DOES work in the steady state (the label is never in last-applied-configuration, so 3-way merge has no claim to remove it), BUT the contract is fragile and inconsistent with the helm template at packages/core/platform/templates/cozystack-version.yaml, which bakes the label inline.
Required fix: replace the imperative-label-then-apply dance with a single declarative apply that includes the label, matching the helm template:
cat <<EOF | kubectl apply --filename -
apiVersion: v1
kind: ConfigMap
metadata:
name: cozystack-version
namespace: ${NAMESPACE}
labels:
platform.cozystack.io/no-delete: "true"
data:
version: "41"
EOFOne source of truth, no last-applied-merge subtlety.
Test that fails without the fix: a bats test (the project already runs bats-unit-tests) that creates a cozy-system/cozystack-version ConfigMap with data.version=40 and no label, runs the migration script, and asserts the resulting ConfigMap has data.version=41 AND metadata.labels["platform.cozystack.io/no-delete"] == "true". Without this, any future migration that forgets the imperative kubectl label step ships green.
3. packages/core/platform/templates/deletion-protection.yaml — the VAP itself has no test coverage
There is zero automated coverage of the actual contract this PR delivers: "a DELETE on an object carrying platform.cozystack.io/no-delete=true is denied with the documented message." This is the e2e-shaped test — a unit test that greps the template is the wrong shape, but an integration test against a real apiserver IS the right shape.
The repo already has packages/core/testing/ with make test. Add a test there that:
- Installs the platform chart on a 1.30+ cluster.
- Asserts
kubectl delete configmap cozystack-version --namespace cozy-systemexits non-zero AND the error message includes "Deletion blocked: object carries platform.cozystack.io/no-delete=true". - Runs
kubectl label configmap cozystack-version --namespace cozy-system platform.cozystack.io/no-delete-, thenkubectl delete, and asserts that succeeds.
Catches every regression the PR is meant to prevent: capability gate inverted, binding objectSelector mistyped, validationActions flipped from Deny to Warn, expression flipped from false to true, label key drift between the binding and the manifests.
Test that fails without the fix: the test above.
4. Documentation drift — the bypass instruction omits the namespace flag
packages/core/platform/templates/deletion-protection.yaml:11packages/core/platform/templates/deletion-protection.yaml:33-35(CELmessage)
Bypass: remove the label, then delete.
kubectl label <kind> <name> platform.cozystack.io/no-delete-
The bypass command is missing --namespace <ns> for namespaced resources. Of the protected set, several are namespaced:
cozystack-versionConfigMap (cozy-system)cozystack-packagesRepository (cozy-system)tenant-rootHelmRelease (tenant-root)
If the operator runs the documented bypass against their default context, kubectl returns "configmap cozystack-version not found" and they think the bypass mechanism is broken. The CEL message (the thing the user sees the moment their delete is denied) has the same bug.
Required fix: change both the template comment and the CEL message to show the namespace flag, e.g.:
To bypass, first remove the label:
kubectl label <kind> <name> --namespace <ns> platform.cozystack.io/no-delete-
(omit --namespace for cluster-scoped resources)
Test that fails without the fix: a helm-template test that asserts the rendered VAP's spec.validations[0].message contains the literal --namespace. Yes, this is content-on-template testing — justified here because the contract is user-facing help text that is otherwise un-asserted.
5. Recommended to fix — failurePolicy: Fail is dead config for a constant CEL expression
packages/core/platform/templates/deletion-protection.yaml:22
The CEL expression is the literal false — it cannot fail to evaluate (no parameters, no variables, no resolved references). failurePolicy: Fail is harmless but misleading. Either drop the field (it defaults to Fail in admissionregistration.k8s.io/v1) or add a comment noting it's redundant.
Test that fails without the fix: none — readability nit.
Reminder: "Pre-existing" is not a thing. If a problem is flagged in this review, it must be fixed in this PR.
Every issue listed above must have a corresponding test. Untested fixes are not fixes.
Aleksei Sviridkin (lexfrei)
left a comment
There was a problem hiding this comment.
Independent second-look — concurring with Timofei Larkin (@lllamnyp)'s NOT LGTM. Adding three observations that don't duplicate the existing blockers, plus a disproven hypothesis worth recording so it doesn't come back later.
Concurring: Timofei Larkin (@lllamnyp)'s four blockers (Go-side label test, migration's imperative-label-then-apply, missing VAP integration test, missing --namespace in the bypass instruction) all still apply on b6b0d8a8. The only change since 2026-05-18 is a main merge — none of the blockers are addressed. The migration in question moved from migrations/40 to migrations/42 (to land after #2454 → 41 and #2694 → 40), but the same imperative-label-then-declarative-apply pattern carries over verbatim into migrations/42:26-32. Fix lllamnyp prescribed (single declarative apply with the label baked in) is unchanged.
Independent additions:
-
helm uninstallof the platform chart is now a bricked operation. Once any labeled object is in the cluster — and this PR labels CRDs, ClusterIssuers, thecozy-systemNamespace, thetenant-rootNamespace + HelmRelease,cozystack-versionConfigMap,cozystack-packagesRepository, LinstorCluster, plus the VAP/Binding themselves —helm uninstallwill issue DELETE on each, and DELETE is denied for the labeled ones. There is no documented teardown procedure. For development clusters and disaster recovery this matters. Recommendation: ship a sibling migration (or a documentedmake uninstallrecipe) that strips the label off the protected set first, then uninstalls. Otherwise every developer hittinghelm uninstall cozy-platformto retry an install gets a half-unwound chart, the VAP staying around, and CRDs they can't remove without manualkubectl label. This is not lllamnyp's documentation-drift blocker — that one was about the per-object bypass message; this one is about the missing teardown story for the whole chart. -
Capability gate gives no visibility when it skips. On a pre-1.30 cluster
Capabilities.APIVersions.Has "admissionregistration.k8s.io/v1/ValidatingAdmissionPolicy"evaluates to false and the whole template renders to nothing — no warning, no hint inhelm installoutput. Operators on older clusters silently lose the guardrail and have no way to discover that from the install log. Either render a NOTES.txt warning when the gate fails, or add a chart-level value (e.g.deletionProtection.requireSupport: true) that fails the install instead of skipping silently. This compounds with Timofei Larkin (@lllamnyp)'s blocker 3 (no integration test): if someone runs this on K8s 1.28 by accident, every other manifest applies, the chart "succeeds", and the guardrail is silently absent. -
Self-protection bypass via Binding-first deletion needs to be acknowledged. The VAP and Binding both carry the
no-deletelabel, so each individually requires the bypass dance. But the order matters: removing the label from the Binding and deleting it first leaves the VAP labeled but with no enforcement → VAP becomes deletable. Removing the label from the VAP first works too. Either order works, and that's by design (the guardrail is operational, not adversarial). Worth a one-liner in the deletion-protection.yaml comment header so a future reader doesn't waste time wondering why the symmetric self-protection isn't truly bilateral. Not a blocker.
Disproven hypothesis (recording so it doesn't get re-raised): an automated reviewer flagged matchConstraints.resourceRules[0].resources: ["*/*"] as supposedly matching only subresources, suggesting the guardrail never fires for top-level deletes like kubectl delete configmap. This is incorrect. Verified against k8s.io/apiserver/pkg/admission/plugin/webhook/predicates/rules/rules.go (Matcher.resource() + splitResource()): */* splits into res="*", sub="*", and the operation's empty subresource matches the sub=="*" wildcard branch. Pattern ["*"] would be the bug (matches only when opSub is empty AND requires explicit subresources to be enumerated separately). ["*/*"] is the correct "everything" form. The current template is right on this dimension.
Leaving the verdict at Timofei Larkin (@lllamnyp)'s NOT LGTM — this review is intended to add context, not to flip the gate. Posting as COMMENT rather than another REQUEST_CHANGES to avoid stacking duplicate change-requests on the same PR (cozystack branch protection treats latest review per author as authoritative, but multiple maintainers asking for the same fixes inflates the noise without changing the gate).
|
Timofei Larkin (@lllamnyp) — thanks for the careful pass, every blocker is addressed. Summary of what landed and where: B1 — B2 — migration uses a single declarative apply (572e08a) B3 — VAP contract has integration coverage (6782761) B4 — bypass docs include B5 — redundant Also picked up the merge from main on the way through (b6b0d8a), which is why migration was renumbered 40 → 42 and |
|
Aleksei Sviridkin (@lexfrei) — thanks for the second-look, all three additions landed. Plus the disproven L1 — L2 — pre-1.30 visibility (c1ac16e) Added L3 — sequential self-protection note (57f7f50) Disproven Also picked up the merge from main on the way through (b6b0d8a), which is why the migration was renumbered 40 → 42 and |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/core/platform/hack/unprotect.sh`:
- Around line 45-61: The script currently masks all kubectl failures by
appending "2>/dev/null || true" to the `kubectl get` calls (used when building
`names` and `pairs`), which hides real errors; update both places (the
`names=$(kubectl get "$kind" ... )` block and the `pairs=$(kubectl get "$kind"
--all-namespaces ... )` block) to capture the kubectl exit status and error
output, allow continuing only when the error indicates the resource kind is
unknown (e.g., error text like "the server doesn't have a resource type" or
similar), but for any other non-zero exit code print the kubectl stderr and exit
non-zero (or skip that kind after logging) so RBAC/API/context failures are not
silenced; ensure you still suppress stdout when continuing for unknown kinds and
keep using variables LABEL, NS_KINDS, names, pairs and the existing loops.
In `@packages/core/platform/images/migrations/migrations/42`:
- Line 16: The migration script uses a POSIX shell shebang (#!/bin/sh) but sets
non-portable pipefail in the line containing "set -euo pipefail"; change this to
a portable form by removing pipefail (use "set -eu") or, if you rely on pipefail
semantics, change the shebang to a Bash interpreter (e.g., bash) so "pipefail"
is supported; update the line containing "set -euo pipefail" and/or the shebang
accordingly to ensure the migration runs on systems where /bin/sh is non-Bash.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: e018de93-72e9-4d40-b6e6-d426f79b9f70
📒 Files selected for processing (9)
hack/e2e-install-cozystack.batsinternal/crdinstall/install_test.gopackages/core/platform/Makefilepackages/core/platform/hack/unprotect.shpackages/core/platform/images/migrations/migrations/42packages/core/platform/templates/NOTES.txtpackages/core/platform/templates/deletion-protection.yamlpackages/core/platform/tests/deletion_protection_test.yamlpackages/core/platform/tests/notes_capability_gate_test.yaml
Aleksei Sviridkin (lexfrei)
left a comment
There was a problem hiding this comment.
LGTM — every blocker from my prior comment review and from Timofei Larkin (@lllamnyp)'s NOT LGTM has a dedicated fix commit. Walking through each:
Timofei Larkin (@lllamnyp) B1 (Go test for install.go label stamping): 84c2aaacd test(crdinstall): assert no-delete label survives Install + merge. TestInstall_appliesAllCRDs now reads CRDs back through the fake client and asserts the label is present. TestInstall_preservesExistingLabels exercises the merge path with a pre-existing app.kubernetes.io/managed-by: foo and verifies both labels survive. Covers the exact regression vector lllamnyp called out — drop the loop, the test fails.
Timofei Larkin (@lllamnyp) B2 (migration imperative-label-then-apply): 572e08a32 fix(platform): migration 42 uses a single declarative apply. The heredoc-built manifest now bakes the label inline; one declarative apply, no 3-way-merge subtlety, idempotent. Matches the helm template at templates/cozystack-version.yaml so the steady-state source of truth is unified.
Timofei Larkin (@lllamnyp) B3 (VAP integration test): 6782761a2 test(platform): e2e check that no-delete VAP denies + bypass works adds the bats e2e covering capability-skip on pre-1.30, label-present precondition, deny-path with the documented message including --namespace, and the round-trip bypass + restore. 23c8dab58 adds tests/deletion_protection_test.yaml for the helm-unittest layer (self-label on both VAP and Binding, message regex, binding selector contract, failurePolicy absence). Both layers cover every regression vector lllamnyp enumerated (binding objectSelector mistyped, validationActions flipped Deny→Warn, expression false→true, label-key drift).
Timofei Larkin (@lllamnyp) B4 (--namespace missing in bypass): 23c8dab58 — both the template comment and the CEL message now include kubectl label <kind> <name> --namespace <ns> platform.cozystack.io/no-delete- with an explicit hint about omitting --namespace for cluster-scoped objects. helm-unittest pins both substrings via matchRegex.
Timofei Larkin (@lllamnyp) Nit (failurePolicy redundant): 602685743 fix(platform): drop redundant failurePolicy on no-delete VAP. Field is gone from the spec and a comment in the template header explains why (CEL expression is the literal false, evaluation cannot fail). helm-unittest asserts notExists: spec.failurePolicy.
My N1 (helm uninstall bricked): bc753f676 feat(platform): make unprotect strips no-delete label for teardown + 67f7934e8 fix(platform): do not swallow kubectl errors in unprotect.sh. The make unprotect recipe runs hack/unprotect.sh, which iterates over the cluster-scoped guarded kinds (CRD, ClusterIssuer, Namespace, VAP, Binding, LinstorCluster) and the namespaced ones (ConfigMap, OCIRepository, HelmRelease) and strips the label. Critically, kubectl_get_or_skip_unknown_kind tolerates only "unknown resource type" (so LinstorCluster on non-LINSTOR clusters doesn't fail the script) and is fatal on any other kubectl error (RBAC, wrong context, API unreachable) — exactly the behavior a teardown helper needs to avoid silent partial unprotection. The follow-up commit also keeps stdout (the resource list) and stderr (warnings/errors) separate so unrelated kubectl warnings can't sneak into the loop variable. Solid.
My N2 (capability gate silent skip): c1ac16e38 feat(platform): NOTES.txt warns when no-delete VAP is silently skipped. helm output now ends with a WARNING block on pre-1.30 clusters explaining that the policy was not rendered, the label is still on objects, and that an upgrade to 1.30+ followed by helm upgrade activates the guardrail. tests/notes_capability_gate_test.yaml exercises both branches.
My N3 (sequential self-protection): 57f7f50cb docs(platform): clarify sequential self-protection on no-delete VAP. Template header now spells out that the label on VAP + Binding is operational guidance, not adversarial defense, and that either one can be unlabeled and deleted first — once the Binding is gone the VAP is unenforced, and vice versa. References make unprotect for the whole-chart path.
One operational note before merge: mergeStateStatus is DIRTY — two conflicts with origin/main:
packages/system/cert-manager-issuers/templates/cluster-issuers.yamlpackages/system/cozystack-basics/templates/tenant-root.yaml
Both are files this PR adds the no-delete label to; main received unrelated changes to the same templates. Rebase, then this PR is good to land. CI is green on the current SHA (Build / E2E / pre-commit / Verify generated all pass).
This APPROVE supersedes my prior COMMENT (#4358008995). Timofei Larkin (@lllamnyp)'s CHANGES_REQUESTED from the previous SHA still stands as the gate — cozystack stale-review auto-dismiss is off, so a fresh look from him is needed to unblock the merge.
Timofei Larkin (lllamnyp)
left a comment
There was a problem hiding this comment.
NOT LGTM
Reviewing against target branch: main.
Scope note
The branch merged origin/main (b6b0d8a83), so the raw merge-base diff drags in unrelated work (etcd backup-strategy controller, scorecard, maintainers, postgres). I reviewed only the deletion-protection feature: the VAP/Binding template, cozystack-version.yaml/repository.yaml/NOTES.txt, migration 42, unprotect.sh, the targetVersion bump, the helm unittests, internal/crdinstall/install.go + test, the e2e bats deny check, and the no-delete label additions across tenant-root/cluster-issuers/linstor/installer namespace.
What I verified works: the VAP design is correct (on DELETE the Binding's objectSelector matches the oldObject's labels, so labeled objects are denied; broad matchConstraints + narrow objectSelector is the right split). go test ./internal/crdinstall/... passes. helm unittest packages/core/platform passes 18/18. The VAP matches only DELETE, so helm/flux updates and the migration's own apply are not blocked.
Blocking issues
1. unprotect.sh still swallows kubectl label failures — the exact bug the last commit claimed to fix
packages/core/platform/hack/unprotect.sh:78-82 and :89-92. Commit 67f7934e8 made the kubectl get path fatal via the helper, but the actual mutating kubectl label ... - calls run inside printf | while read pipelines. Under set -eu with no pipefail (dash on CI), a non-zero exit inside a piped while subshell does not abort the script. A failed label removal (RBAC, transient API error) is swallowed and the script still prints "Done. helm uninstall is now unblocked." and exits 0 — leaving the object protected. That is the "report success while leaving labels behind" failure the helper comment calls fatal; it's only guarded on the read path. Fix: iterate without the pipe-subshell (for ref in $names; do kubectl label "$ref" "$LABEL-"; done), or track an rc and exit 1 if any label call failed. Test: a shim where kubectl label exits 1 for one object; assert the script exits non-zero and doesn't print "unblocked".
2. unprotect.sh doesn't cover a GitRepository source
unprotect.sh:63-67 lists ocirepository but not gitrepository. repository.yaml labels a resource of kind {{ $sourceRef.kind }} (configurable; source.toolkit.fluxcd.io/v1 also serves GitRepository). On a GitRepository-sourced cluster the label is never stripped and helm uninstall stays denied, with the script exiting 0. Fix: add gitrepository to NS_KINDS or derive the kind from sourceRef.kind. Test: run against a fixture whose only labeled source is a GitRepository, assert the label is removed.
Recommended (non-blocking)
- Runner silently skips missing migrations.
run-migrations.sh:27-38prints "Migration N not found, skipping" and continues when a migration file belowTARGET_VERSIONis absent. In a shipped release this is harmless — the release build (make image-migrations) rebuilds the migrations image from the same source tree that carries the migration files and restamps the digest in the same commit, so image andtargetVersionare always consistent at release time; the digest invalues.yamlis a generated artifact, never hand-set in feature PRs. But hardening the runner to hard-error on a missing migration belowTARGET_VERSION(instead of skipping) would turn a future packaging mistake into a loud failure rather than a silent version stall. Out of scope for this PR (it doesn't touch the runner), noting for a follow-up. - Migration 42's "no 3-way-merge subtlety" comment is slightly overstated (prior migrations stamp the version with a label-less apply, so a client-side last-applied interaction does exist across migrations) — correct in outcome, not worth blocking.
- The e2e bypass-path teardown re-applies
cozystack-versionwithout thehelm.sh/resource-policy: keepannotation the chart bakes in, so it doesn't fully restore cluster state. cluster-issuers.yamlhas pervasive trailing whitespace on lines this PR touched; asedcleanup pass would be welcome.
Reminder: "Pre-existing" is not a thing. If a problem is flagged in this review, it must be fixed in this PR — no exceptions.
Every issue listed above must have a corresponding test. No matter the severity — bug, logic error, security issue, error handling gap, documentation drift — each one must be covered by a test that fails without the fix and passes with it. Untested fixes are not fixes.
Sources for the VAP DELETE/objectSelector behavior I verified: ValidatingAdmissionPolicyBinding | Kubernetes, Validating Admission Policy | Kubernetes.
…sionPolicy Block accidental DELETE on critical platform objects (tenant-root, cozy-system, LinstorCluster, CRDs, ClusterIssuers, cozystack-version ConfigMap, cozystack-packages OCIRepository, tenant-root HelmRelease) via a label-scoped ValidatingAdmissionPolicy. No webhook DaemonSet, Service, TLS, or image — the API server evaluates the CEL expression in-process. Requires Kubernetes 1.30+. Bypass for legitimate deletes is two steps: kubectl label <kind> <name> platform.cozystack.io/no-delete- kubectl delete <kind> <name> CRDs installed dynamically by the operator get the label stamped in internal/crdinstall/install.go before the apply call. Release note: Added deletion-protection guardrail (ValidatingAdmissionPolicy) blocking DELETE on cozystack platform objects labeled platform.cozystack.io/no-delete=true. Bypass by removing the label first. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
…bility Address review feedback from gemini-code-assist on packages/core/platform/templates/deletion-protection.yaml:42: wrap the ValidatingAdmissionPolicy and Binding in a Helm capability check so the template stays renderable on clusters that pre-date 1.30 (when ValidatingAdmissionPolicy graduated to GA). The check targets the specific Kind, not the API group/version — admissionregistration.k8s.io/v1 itself has existed since 1.16 for ValidatingWebhookConfiguration, so the group/version alone is not a meaningful capability gate. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from gemini-code-assist on packages/core/platform/templates/deletion-protection.yaml:15: label the ValidatingAdmissionPolicy and ValidatingAdmissionPolicyBinding themselves with platform.cozystack.io/no-delete=true so the guardrail guards itself. A direct kubectl delete on either resource now returns the same denial as deletion of any other protected object; the same two-step label-then-delete bypass applies when intentional removal is needed. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
unprotect.sh: add `gitrepository` to the namespaced-kind list so the
platform stays unprotectable when an operator configures
`sourceRef.kind: GitRepository`. Currently only `ocirepository` is
swept, while repository.yaml templates `kind: {{ $sourceRef.kind }}`
unrestricted.
run-migrations.sh: replace the silent skip on a missing migration file
with a hard error. All migrations 1..targetVersion exist in the shipped
image — a gap can only mean a packaging mistake (digest advanced
without the file landing), which previously stalled the cluster at a
stale version while reporting success.
migrations/42: drop the misleading "no 3-way-merge subtlety" claim from
the header. The apply records the no-delete label under the default
field manager's last-applied annotation, so a future label-less apply
by the same field manager would strip it. Document the constraint
instead of denying it.
hack/e2e-install-cozystack.bats: restore `helm.sh/resource-policy: keep`
on the reconstructed cozystack-version ConfigMap in the bypass-path
teardown so cluster state matches the chart template shape
(packages/core/platform/templates/cozystack-version.yaml) on exit.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
67f7934 to
8fb4d50
Compare
…ersion
Address review feedback from coderabbitai on
packages/core/platform/templates/cozystack-version.yaml:10:
existing clusters skipped the label because the template's
`lookup`+`if not $configMap` guard never re-renders the ConfigMap on
upgrade (the skip is intentional — it prevents helm from clobbering the
data.version that the migration controller writes from its pre-upgrade
hook).
A one-shot migration is the correct mechanism here:
- Migration 42 in packages/core/platform/images/migrations/migrations/42
runs `kubectl label --overwrite configmap cozystack-version
platform.cozystack.io/no-delete=true` and stamps version=43. Slot 42
rather than 40 because main already occupies 40 (dashboard CRD
cleanup) and 41 (kubernetes ephemeralStorage→diskSize rename).
- Bump migrations.targetVersion 42 -> 43 so the hook fires on the
next platform upgrade.
Idempotent: --overwrite is a no-op when the label is already at the
target value, MIGRATION_DRY_RUN=1 prints without applying, and the
script aborts before stamping if the kubectl label call fails.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from @lllamnyp on internal/crdinstall/install.go: the deletion-protection label loop was untested — TestInstall_appliesAllCRDs only checked that Install returned no error, so a refactor dropping the loop would have shipped green and silently disabled VAP coverage on every Cozystack CRD. - Extend TestInstall_appliesAllCRDs to read each applied CRD back from the fake client and assert platform.cozystack.io/no-delete=true is present. - Add TestInstall_preservesExistingLabels: feeds in a CRD whose manifest already carries app.kubernetes.io/managed-by=foo, asserts both that label and the deletion-protection label end up on the applied object. Locks down the merge contract in install.go:92-101 (the GetLabels()==nil branch is still covered by the first test). Verified the new assertions fail when the label-stamping loop is removed. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from @lllamnyp on packages/core/platform/images/ migrations/migrations/42: the previous version did `kubectl label --overwrite` then `kubectl apply` on a manifest with no labels, relying on `kubectl apply`'s 3-way strategic merge to leave the imperatively-set label intact. That works in the steady state but is fragile — the contract is "the label is preserved because it is never in last-applied-configuration" — and it is inconsistent with the helm template at templates/cozystack-version.yaml, which bakes the label inline. Replace the two-step dance with one declarative apply that includes the deletion-protection label, matching the chart template. One source of truth, no last-applied-merge subtlety. The script remains idempotent (apply is a no-op once the ConfigMap matches). Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from @lllamnyp on packages/core/platform/templates/ deletion-protection.yaml: the VAP's validation expression is the literal `false` — no parameters, no variables, no references — so CEL evaluation cannot fail and the api-server never has occasion to apply failurePolicy. The previous `failurePolicy: Fail` line was a no-op masquerading as operational config. Drop the field (the default is Fail anyway in admissionregistration.k8s.io/v1) and document in the template comment why it is intentionally absent, so the next reader does not re-add it on the assumption that it was forgotten. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from @lllamnyp on packages/core/platform/templates/ deletion-protection.yaml: The bypass instructions in both the template-author-facing comment and the user-facing CEL message did `kubectl label <kind> <name>` with no --namespace flag. Several protected objects are namespaced (cozystack-version ConfigMap, cozystack-packages Repository, tenant-root HelmRelease), so an operator who copies the documented command verbatim against their default context gets a "not found" error and concludes that the bypass mechanism is broken. Updated both call sites to show --namespace <ns> with an explicit "(omit for cluster-scoped resources)" caveat. The reviewer also flagged that the VAP contract is otherwise unasserted by unit tests. Add packages/core/platform/tests/deletion_protection_test.yaml covering: - both the VAP and the Binding carry platform.cozystack.io/no-delete=true on themselves (the self-protection contract) - failurePolicy is intentionally absent on the VAP - the deny message contains the bypass command WITH --namespace and the cluster-scoped caveat (regression catch for this fix; verified failing when --namespace is dropped from the template) - the Binding's objectSelector + validationActions: [Deny] still match the no-delete label Wire packages/core/platform/Makefile's test target to helm unittest so hack/helm-unit-tests.sh actually runs the new suite in CI (the existing tests/ files were orphaned without it). Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from @lllamnyp on packages/core/platform/templates/ deletion-protection.yaml: the VAP contract had no integration coverage. A unit test that greps the rendered manifest is the wrong shape — the contract is "the api-server denies a labeled DELETE with the documented message" and that needs a real apiserver in the loop. Add a post-install bats test to hack/e2e-install-cozystack.bats that runs once cozystack is installed on the e2e cluster: 1. Skips on clusters that pre-date Kubernetes 1.30 (no VAP API). 2. Asserts the cozystack-version ConfigMap actually carries platform.cozystack.io/no-delete=true — precondition, so the deny assertion below would not misreport on a regressed binding. 3. Asserts `kubectl delete configmap cozystack-version -n cozy-system` exits non-zero AND its error message contains the documented "Deletion blocked: ... platform.cozystack.io/no-delete=true" substring AND the `--namespace` bypass hint. 4. Confirms the ConfigMap survives the rejected delete. 5. Removes the label, deletes (must succeed), recreates the ConfigMap with the original data.version and the label so the cluster ends the test in the same state it started. This single test catches every regression the PR is meant to prevent: capability gate inverted, binding objectSelector mistyped, validationActions flipped Deny→Warn, expression flipped false→true, label-key drift between the binding and the manifests, and the bypass docs missing --namespace. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from @lexfrei on packages/core/platform/templates/ deletion-protection.yaml: both the ValidatingAdmissionPolicy and the ValidatingAdmissionPolicyBinding carry the no-delete label so each individually requires the bypass dance, but the labeling is symmetric, not truly bilateral. Removing the label from the Binding and deleting it first leaves the VAP labeled but unenforced, so the VAP itself is then deletable without the bypass — and vice versa. That is by design: the guardrail is operational guidance, not adversarial defense against an operator with cluster-admin who is determined to remove it. Add a paragraph in the template comment header so a future reader does not waste time wondering why the symmetric self-protection isn't truly bilateral. Forward-references the whole-chart teardown recipe (make unprotect) so the comment links to the documented escape hatch. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from @lexfrei on packages/core/platform/templates/ deletion-protection.yaml: on a pre-1.30 cluster the capability gate (`.Capabilities.APIVersions.Has "admissionregistration.k8s.io/v1/ ValidatingAdmissionPolicy"`) evaluates to false and the policy + Binding render to nothing. There was no signal in `helm install`/`helm upgrade` output that the guardrail had been silently skipped, so operators on older clusters lost the protection without any way to discover that from the install log. Compounded the integration-test gap @lllamnyp called out in blocker 3. Add packages/core/platform/templates/NOTES.txt that renders a multi-line WARNING whenever the capability is absent: states the guardrail is NOT active, explains why (1.30+ required), notes that the label is still stamped on every protected object so the guardrail becomes active automatically on the next upgrade after a cluster bump, no migration step required. Add packages/core/platform/tests/notes_capability_gate_test.yaml: positive case asserts the warning text + the 1.30 hint + the label string render on a chart-level `capabilities.apiVersions: []`. Negative case asserts the warning is absent when admissionregistration.k8s.io/v1/ ValidatingAdmissionPolicy is in the capability list. Verified the positive case fails when the warning text is removed from NOTES.txt. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from @lexfrei on packages/core/platform/templates/ deletion-protection.yaml: with this PR every protected object — CRDs, ClusterIssuers, cozy-system and tenant-root Namespaces, cozystack-version ConfigMap, cozystack-packages Repository, tenant-root HelmRelease, LinstorCluster, plus the VAP and Binding themselves — carries the no-delete label. `helm uninstall cozy-platform` issues DELETE on each, and DELETE is denied for every one of them, so the chart effectively cannot be uninstalled without manual `kubectl label` against each object. Disaster recovery and dev-cluster reset both regress as a result. Add packages/core/platform/hack/unprotect.sh: enumerates the guarded set locally (per-kind, not via a single all-resources sweep — some kinds may not exist on every install, e.g. linstorcluster on a non-LINSTOR cluster, and kubectl errors on unknown kinds). For cluster-scoped kinds, lists objects with the label and labels them off one at a time. For namespaced kinds, lists (namespace, name) pairs via jsonpath and labels each in the right namespace. Idempotent: a label that is already absent is a no-op. Wire `make unprotect` in packages/core/platform/Makefile so the operator runs it before `helm uninstall`. The Makefile target carries a comment noting this is teardown / disaster-recovery only — after it runs, the cluster has no guardrail until the next `helm upgrade` re-stamps the label. Verified with a stub kubectl in PATH that the script issues the right get/label calls for both cluster-scoped (e.g. `kubectl label customresourcedefinition/packages.cozystack.io platform.cozystack.io/ no-delete-`) and namespaced kinds (e.g. `kubectl label --namespace tenant-root helmrelease tenant-root platform.cozystack.io/no-delete-`). The template-header reference at templates/deletion-protection.yaml that forward-links to `make unprotect` was added in the preceding L3 commit; the recipe it points at now exists. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The blanket `2>/dev/null || true` suppression masked RBAC, wrong-context, and API-unreachable failures, letting the script print "Done." while leaving labels in place. Replace with a helper that only tolerates the "unknown resource type" error (the actual best-effort case — kinds that don't exist on every cluster, e.g. linstorcluster on a non-LINSTOR install) and propagates everything else. Also keep stdout (resource list) and stderr (kubectl warnings) separate so deprecation warnings can't sneak into the loop variable. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
unprotect.sh: add `gitrepository` to the namespaced-kind list so the
platform stays unprotectable when an operator configures
`sourceRef.kind: GitRepository`. Currently only `ocirepository` is
swept, while repository.yaml templates `kind: {{ $sourceRef.kind }}`
unrestricted.
run-migrations.sh: replace the silent skip on a missing migration file
with a hard error. All migrations 1..targetVersion exist in the shipped
image — a gap can only mean a packaging mistake (digest advanced
without the file landing), which previously stalled the cluster at a
stale version while reporting success.
migrations/42: drop the misleading "no 3-way-merge subtlety" claim from
the header. The apply records the no-delete label under the default
field manager's last-applied annotation, so a future label-less apply
by the same field manager would strip it. Document the constraint
instead of denying it.
hack/e2e-install-cozystack.bats: restore `helm.sh/resource-policy: keep`
on the reconstructed cozystack-version ConfigMap in the bypass-path
teardown so cluster state matches the chart template shape
(packages/core/platform/templates/cozystack-version.yaml) on exit.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
8fb4d50 to
76fd5d1
Compare
Aleksei Sviridkin (lexfrei)
left a comment
There was a problem hiding this comment.
LGTM on 76fd5d179. Walked through Timofei Larkin (@lllamnyp)'s second-round review against this SHA: blocker B2 + all 4 recommended items are landed; blocker B1 is empirically a false positive — the pipe-subshell does abort under set -eu, against the migrations image's actual shell (busybox ash on alpine 3.23).
Timofei Larkin (@lllamnyp) blocker check
B1 (unprotect.sh pipe-subshell allegedly swallows kubectl label failures) — DISPROVEN
The claim was: "under set -eu with no pipefail (dash on CI), a non-zero exit inside a piped while subshell does not abort the script. A failed label removal ... is swallowed and the script still prints 'Done. helm uninstall is now unblocked.' and exits 0."
Tested against the four interpreters the script could meet on the rendered surface:
# alpine 3.23 (migrations image), shell is BusyBox v1.37.0 ash
$ docker run --rm alpine:3.23 sh -c '
set -eu
printf "a\nb\nc\n" | while read x; do
counter=$((${counter:-0} + 1))
if [ "$counter" = "2" ]; then
echo "kubectl label failed on iteration $counter" >&2
false
else
echo "labeled object iteration $counter"
fi
done
echo "still alive (BAD)"'
# → labeled object iteration 1
# → kubectl label failed on iteration 2
# → exit 1, "still alive" NEVER prints
Same result on dash (POSIX strict), on bash without pipefail, and on bash with pipefail. Mechanism: set -e inside the while-subshell triggers on the first non-zero (false or kubectl label's exit code), the subshell aborts with that code, the pipeline's last command is the subshell so the pipe exits non-zero, and the outer shell's set -e then propagates. POSIX rule "set -e is ignored in command lists immediately following while/until" applies to the condition, not the body — man bash is explicit. So the unprotect.sh structure on this SHA does fail loud on a kubectl label error rather than silently exiting 0.
There's a narrow scenario where lllamnyp's claim would be true — if the kubectl label line were on the last logical iteration AND the pipeline's exit code were captured into a variable ($(... ) or || true) — but the current script does neither. The bare printf | while read; do ... ; done form propagates correctly.
B2 (unprotect.sh missing gitrepository) — ADDRESSED in 76fd5d179
packages/core/platform/hack/unprotect.sh:64-72 now lists gitrepository alongside ocirepository under NS_KINDS, with the new comment correctly attributing it to repository.yaml templating kind: {{ $sourceRef.kind }} unrestricted. On a GitRepository-sourced cluster, helm uninstall is no longer left denied.
Timofei Larkin (@lllamnyp) recommended items
- Runner silently skipped missing migrations —
run-migrations.sh:25-44now hard-errors with"Migration $i not found in image — refusing to advance past a missing migration"andexit 1instead of skipping. Daniil went above the recommended bar — actually flipped the default from "silent skip" to "loud fail" with a release-build-consistency comment explaining why. ✓ - Migration 42 "no 3-way-merge subtlety" comment overstated — comment now clarifies the actual constraint: future migrations re-stamping
cozystack-versionunder the same field manager MUST include the label inline. ✓ - e2e bypass-path teardown missed
helm.sh/resource-policy: keep—hack/e2e-install-cozystack.bats:340-356now bakes the annotation into the reconstruct apply, restoring the same shape the chart template produces. ✓ - Trailing whitespace in
cluster-issuers.yaml— not addressed in76fd5d179. Pure cosmetic, not a blocker.
Net
This APPROVE supersedes my prior APPROVE (#4361721935) that was auto-dismissed during the rebase, and addresses the new gate that Timofei Larkin (@lllamnyp) added on 67f7934e — B2 + recommended items landed in 76fd5d179, and B1 doesn't reproduce against the actual shells in play.
Gate-wise: Timofei Larkin (@lllamnyp)'s CHANGES_REQUESTED is still active on the prior SHA, and cozystack auto-dismiss is off — needs a fresh look from him to unblock the merge.
The deletion-protection commit added a second, identical 'test:' recipe to the platform Makefile, causing GNU Make to print 'overriding recipe' warnings on every 'make test' (and thus in helm-unit-tests CI output). Drop the duplicate; keep the original target. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Timofei Larkin (lllamnyp)
left a comment
There was a problem hiding this comment.
LGTM
The deletion-protection feature is sound and well-tested: the VAP design is correct (broad matchConstraints on DELETE narrowed by the binding's objectSelector on platform.cozystack.io/no-delete), helm unit tests pass 24/24, go test ./internal/crdinstall/... passes, and there's a real e2e deny+bypass test wired into CI via make -C packages/core/testing install-cozystack.
The one blocker from the prior round — a duplicate test: target in packages/core/platform/Makefile that made GNU Make print overriding recipe warnings on every make test (and thus in helm-unit-tests CI output) — is fixed in 3bac5caac. make -C packages/core/platform -n test now runs clean.
Non-blocking observations (author's discretion)
unprotect.shdoesn't cover allsourceRef.kindvalues.repository.yamltemplateskind: {{ $sourceRef.kind }}unrestricted, butunprotect.sh'sNS_KINDSlists onlyocirepository/gitrepository.source.toolkit.fluxcd.io/v1also admitsHelmRepositoryandBucket. Default isOCIRepositoryso it's fine in practice, but a non-defaultsourceRef.kindwould leave that object labeled and blockhelm uninstall. Deriving the kind fromsourceRef.kindwould close the gap.- e2e recovery fragility on retry. The new bats test strips the label, deletes
cozystack-version, then reconstructs it, and the CI step wraps the file in anuntil attempt<3retry. An abort between strip and reconstruct leaves thegrep -qx 'true'precondition failing on retry with no recovery. Low-risk (likely abort points are assertions that already recorded the failure), but ateardown()/trap re-stamp would make the retry meaningful.
Neither blocks merge.
Empty commit to fire the Pull Request workflow, which never ran for 3bac5cac because GitHub Actions was down during the original push. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
What this PR does
Adds a
ValidatingAdmissionPolicythat blocksDELETEon critical platformobjects labeled
platform.cozystack.io/no-delete=true. Native, in-process CELevaluation — no webhook DaemonSet, no Service, no TLS, no image to maintain.
Requires Kubernetes 1.30+.
This is an alternative to the closed #2402 (admission webhook approach).
Protected objects (labeled in this PR)
cozy-systempackages/core/installer/templates/cozy-system-{namespace,labels}.yamltenant-rootpackages/system/cozystack-basics/templates/tenant-root.yamltenant-rootcozystack-versionpackages/core/platform/templates/cozystack-version.yamlcozystack-packagespackages/core/platform/templates/repository.yamlletsencrypt-prod,letsencrypt-stage,selfsigned-cluster-issuerpackages/system/cert-manager-issuers/templates/cluster-issuers.yamllinstorclusterpackages/system/linstor/templates/cluster.yamlpackages.cozystack.io,packagesources.cozystack.iointernal/crdinstall/install.goBypass
Why VAP instead of a webhook
Same outcome as #2402 but with zero runtime infrastructure: no DaemonSet, Service,
TLS certificate, or image. The kube-apiserver evaluates the CEL
expression: "false"in-process and denies becausevalidationActions: [Deny].objectSelectoron the binding scopes evaluation to labeled objects only — everyother DELETE in the cluster is unaffected.
Release note
Summary by CodeRabbit
New Features
Chores
Tools
Documentation
Tests