v1.6.2 (2026-08-19)

A patch release with six fixes covering the backup-strategy controller, kube-ovn's webhook certificate, Velero CRD upgrades, CNPG barman-cloud backups, flux-shard-operator, and the published OpenAPI definitions, plus a release-pipeline reliability fix.

Fixes

• fix(backupstrategy-controller): repair lookup-gated backup objects: The default backup Strategy CRs and the Velero BackupStorageLocation are gated on a Helm lookup performed while the referenced object is still being created; when that lookup came back empty the objects were skipped permanently, since helm-controller does not re-render a release whose chart and values are unchanged. The gate now resolves the default bucket credentials Secret through the RESTMapper, bounds each check, and tolerates an absent Secret instead of looping, so the default backup objects are created reliably instead of silently vanishing for months (@mattia-eleuteri in #3524, backport #3731).

• fix(kube-ovn): reload kubeovn-webhook serving certificate on cert-manager renewal: kube-ovn-webhook loaded its TLS serving certificate once at startup and never re-read it; once cert-manager renewed the backing Secret and the old certificate expired, the apiserver's calls to the webhook failed verification and, because the MutatingWebhookConfiguration uses failurePolicy: Fail, every pod creation in tenant namespaces was rejected — including virt-launcher pods, blocking VMI startup. The webhook now serves its certificate through a reloading callback that re-reads the key pair when the mounted files change and widens renewBefore to 720h, so cert-manager renewals are honored without a pod restart (@IvanHunters in #3557, backport #3730).

• fix(velero): apply CRD updates on upgrade via CreateReplace: Velero's CRDs stayed frozen at whatever version was first installed, since Helm never touches a chart's crds/ directory on upgrade; when the Velero image moved to a version that added new backup phases, the apiserver rejected phase transitions against the stale CRDs and backups silently stopped while the HelmRelease stayed green. The Velero package now opts into upgradeCRDs: CreateReplace, so CRDs are kept current on upgrade and backups keep working (@lexfrei in #3727, backport #3728).

• fix(backups): request S3 checksum only when required for barman-cloud (non-AWS S3 / Ceph RGW): CNPG's barman-cloud plugin sidecar defaulted to computing a flexible checksum on every upload, which several S3-compatible backends (Ceph RGW, some MinIO / Cloudflare R2 builds) reject outright, so every backup and WAL-archive upload to those backends failed and ScheduledBackups never stored anything. Every barman-cloud ObjectStore Cozystack creates — Keycloak's system DB, the postgres app's backup and recovery stores, and the platform-managed system-bucket store — now sets AWS_REQUEST_CHECKSUM_CALCULATION=when_required, a safe default accepted by both AWS S3 and the affected backends (@androndo in #3417, backport #3767).

• fix(flux-shard-operator): repair sharded helm-controller crashloop behind an HTTP proxy: The cloned helm-controller-shard<i> Deployment inherited HTTP_PROXY/HTTPS_PROXY/NO_PROXY from the flux-aio all-in-one wiring even though a standalone shard needs no external egress; behind an unreachable proxy the controller's blocking startup HTTPS call never completed, the manager never served /healthz, and every HelmRelease sharded to that controller was frozen. The sanitisation now also drops the inherited proxy env and adds a startupProbe derived from the liveness handler, so sharded HelmReleases keep reconciling in proxied environments instead of crashlooping forever (@IvanHunters in #3546, backport #3818).

• fix(api): declare OpenAPIModelName for core and sdn types: The core and sdn API groups did not declare OpenAPIModelName the way the apps group already did, so their published OpenAPI definition names were Go import paths while every $ref pointing at them escaped each slash — the two spellings never matched, the reference dangled, and kubectl apply --validate failed on any resource against a cozystack-api built after the underlying Kubernetes 0.35 change. Declaring OpenAPIModelName for core and sdn too makes every published definition name the dotted Kubernetes model name, so client-side validation against the published OpenAPI works again (@myasnikovdaniil in #3808, backport #3812).

Development, Testing, and CI/CD

• ci(release): complete the candidate-aware promotion pipeline on release-1.6: release-1.6 was missing the e2e and packages-verification jobs that Promote RC requires on its target base, so v1.6.1 was promoted with the rc e2e gate bypassed and the next patch release could not even be dispatched. Adds the rc-e2e job, the verify-release-candidate checks, hack/verify-promoted-packages.sh, hack/validate-changelog.sh and regression tests pinning the pipeline's contract, so future patch releases off release-1.6 run the same e2e and package-verification gates as main before promoting, and the tag-time changelog is validated and ported from the tag rather than regenerated (@myasnikovdaniil in #3893).

Documentation

• [website] docs: import the operator guides that lived in the cozystack repo: Moves the operator-facing guides that used to live in the cozystack repo over to the documentation site, so operators find them alongside the rest of the docs instead of scattered across two repositories (@myasnikovdaniil in cozystack/website#648).

• [website] docs(oidc): document private CA and staging trust: Documents how to configure tenant OIDC to trust a private certificate authority and staging certificates, closing a gap for operators running their own CA or testing with a staging issuer (@myasnikovdaniil in cozystack/website#650).

• [website] feat(community): add a Community page and link it from the main menu: Adds a Community page linked from the site's main menu, giving visitors a single place to find how to get in touch with and contribute to the Cozystack community (@tym83 in cozystack/website#637).

• [website] chore(telemetry): publish July 2026 and explain how the figures are derived: Publishes the July 2026 telemetry figures and documents how those figures are derived, giving the community visibility into adoption trends and how the numbers are calculated (@tym83 in cozystack/website#644).

• [website] feat(blog): new Blockstor banner: Adds a new banner promoting Blockstor to the blog, improving the visibility of the storage control plane's announcement (@tym83 in cozystack/website#646).

Contributors

Thanks to everyone who contributed to this patch release:

• @IvanHunters
• @androndo
• @lexfrei
• @mattia-eleuteri
• @myasnikovdaniil
• @tym83

Full Changelog: v1.6.1...v1.6.2

v1.5.4 (2026-08-19)

v1.5.4 is the final release of the 1.5 line. It is a stability release: it backports fixes for a webhook certificate-renewal outage that could block all pod creation, a KubeVirt VMI validation failure, several release-blocking crashloops (velero, cert-manager, SeaweedFS, flux-shard-operator), a silent PostgreSQL restore data-integrity bug, and a raft of smaller reliability and CI fixes accumulated on the branch. It also closes the SeaweedFS 4.31 rename fallout on the 1.5.x line and pins the CAPI kubeadm bootstrap objects that a later upgrade to 1.6 would otherwise prune — both of which need an operator to act, and both of which are covered in the section below.

These notes are measured against v1.5.2, not v1.5.3. v1.5.3 was tagged but its GitHub Release was left a draft and never published, so no user ever received it and every operator upgrading arrives from v1.5.2. Comparing against v1.5.3 would silently drop four commits — two user-facing fixes — that nobody has seen in a release. The range is v1.5.2..v1.5.4, 82 commits across 27 pull requests.

⚠️ Breaking Changes and Required Actions

There are no breaking API or values changes in v1.5.4. There are two things that need an operator, and both of them can cost data or wedge an upgrade if they are skipped. Read this section in full before applying the v1.5.4 Platform Package.

Pre-upgrade checks

Run these against the management cluster before upgrading.

1. SeaweedFS 4.31 rename — classify every instance, and re-run the audit even if you have run it before.

Cozystack v1.5.0 bumped the vendored SeaweedFS chart from 4.0.405 to 4.31.0. Before 4.31 the chart named its workloads after the chart (seaweedfs-master, seaweedfs-filer, seaweedfs-volume), ignoring the Helm release name. 4.31 names them after the release, and the data-plane release is <name>-system, so every StatefulSet wanted to become seaweedfs-system-*. StatefulSet names are immutable, so Helm could not rename in place — it stood up a second, duplicate set beside the running one.

What that duplicate does depends on the cluster. With as many nodes as master replicas the new masters cannot schedule (hard pod anti-affinity against the old ones), so the duplicate sits Pending/CrashLoopBackOff and the original keeps serving. With more nodes than masters the new, empty set comes up — and because both sets carry identical pod labels, the seaweedfs-s3 Service load-balances across them while both filers write to the same seaweedfs-db Postgres metadata store pointing at different volume servers. That is a data-integrity incident, not a cosmetic duplicate: reads of existing objects through the new endpoint miss, new writes land on empty volumes, and two master sets hand out volume IDs from independent sequences into one shared metadata table.

Separately, and landing on the same upgrade, the v1.5.0 database split moved the CNPG Cluster/seaweedfs-db — the filer metadata store, i.e. the index for every object in the tenant's S3 — out of the <name>-system release into its own <name>-db release. Migration 43 shipped comparing the owning release against the literal string seaweedfs-system, so it only ever fired for an instance named seaweedfs; an instance named anything else was skipped and had its Cluster pruned as a removed resource, with CNPG taking the PVC along with it. That prune is not a one-shot: Helm computes deletions by diffing the last deployed revision against the new manifest, so a tenant whose <name>-system last succeeded on a pre-split revision recomputes the same deletion on every upgrade attempt, including attempts that fail for unrelated reasons.

v1.5.4 pins fullnameOverride: seaweedfs in system/seaweedfs, so workloads are named after the chart exactly as they were before 4.31 and upgrading adopts the running set and its volumes in place. Migration 43 is fixed to match the -system suffix, and migration 45 re-runs the hand-over for clusters that already ran the hardcoded version. Two states cannot be adopted that way, and the chart fails the render rather than guess — the enforcing guard is packages/system/seaweedfs/templates/naming-guard.yaml, with a sibling copy in extra/seaweedfs so the refusal is visible on the SeaweedFS application itself.

Step 0 — seaweedfs-db ownership (read-only, do this first). This one destroys data rather than duplicating it, so clear it before anything else.

kubectl get cluster.postgresql.cnpg.io -A \
  -o custom-columns='NS:.metadata.namespace,NAME:.metadata.name,OWNER:.metadata.annotations.meta\.helm\.sh/release-name,KEEP:.metadata.annotations.helm\.sh/resource-policy'

Read the rows where NAME is seaweedfs-db:

Step 1 — classify every SeaweedFS instance (read-only, mutates nothing).

hack/seaweedfs-naming-audit.sh                 # whole cluster
hack/seaweedfs-naming-audit.sh tenant-foo      # or named namespaces

Read the exit code, not just the table. The audit fails closed: a kubectl call that fails, or a Helm release payload it cannot decode, prints FATAL and exits non-zero, and the partial table must not be trusted. Only exit 0 means the table is the whole answer — and an empty table with exit 0 is a genuinely clean fleet.

If you have already run this audit, run it again on v1.5.4. The version of the script that shipped in v1.6.0 silenced every kubectl failure with 2>/dev/null, so a timeout or an RBAC denial produced an empty, "all clean" table byte-identical to an honestly clean fleet — a false clean, on the script whose output gates a runbook step that deletes PVCs. v1.5.4 is the first release on the 1.5 line to carry the audit at all, and it carries the fail-closed version (v1.6.1 and later carry it too). A clean result from a v1.6.0 checkout, or from main between 2026-07-20 and 2026-07-28, is not evidence of anything.

One unrelated filer change lands on the same upgrade and is worth knowing about while you are looking at this: the filer's postgres2 connection pool to that same seaweedfs-db metadata store was unconfigured, so every metadata lookup opened a fresh PostgreSQL connection and added seconds of latency to every S3 request. That is fixed here too (see the postgres2 connection pool entry below), and it needs no operator action — but if you have been treating slow S3 as a symptom of the rename, it may well have been this instead.

Recovery for S and MIXED is docs/operations/seaweedfs-431-rename-recovery.md. Do not guess which generation holds the data — the runbook exists because a duplicate that briefly served writes and later crashed is indistinguishable, on every durable signal, from one that never scheduled. A tenant that went 1.4.x straight to 1.6 never renamed and is unaffected by any of this; duplicates exist only on tenants that passed through 1.5.x.

2. The platform migration targetVersion moves from 45 to 46 — which changes what a later upgrade to 1.6 runs.

v1.5.4 is the first 1.5.x release stamped targetVersion: 46; v1.5.0 through v1.5.3 were all stamped 45. run-migrations.sh loops seq CURRENT (TARGET - 1), so a cluster that reaches 46 and later upgrades to v1.6 (targetVersion: 54) runs slots 46 through 53 — and never executes 1.6's own slot 45. This is a skip, not an ordering problem.

Slot 45 does not hold the same thing on both branches. On release-1.5 it is the SeaweedFS seaweedfs-db hand-over repair described above. On main and release-1.6 it is the pin that stamps helm.sh/resource-policy: keep onto the CAPI KubeadmConfigTemplate objects. 1.6 drops KubeadmConfigTemplate from the tenant kubernetes chart entirely — workers move to TalosConfigTemplate — so on that upgrade Helm sees the object in the previous release manifest, absent from the new one, and deletes it while the kubeadm-backed MachineSet is still mid-rollover with its bootstrap.configRef pointing at it. controller-manager then floods with reconcile errors, and where the Talos image fetch is slow or a MachineHealthCheck remediates, workers can han...

v1.6.1 (2026-08-05)

A patch release with seven fixes covering PostgreSQL, etcd, managed Kubernetes, Keycloak, cozystack-basics, and SeaweedFS, plus release-pipeline reliability fixes and a talm update adding declarative Talos preset knobs.

Fixes

• fix(postgres-operator): align CNPG operator and CRDs to 1.28.2 for PVC resize-deadlock fix: A simultaneous resources + size change on a single-instance PostgreSQL cluster could make the CloudNativePG operator delete the sole primary Pod, classify the PVC as resizing, and never recreate the Pod — wedging the cluster with zero instances and leaving the filesystem resize incomplete. Bumping the operator image and CRDs together to 1.28.2 (which carries upstream's fix, cloudnative-pg#9980 / cloudnative-pg#9981) resolves the deadlock (@scooby87 in #3510, backport #3542).

• chore(etcd-operator): bump etcd-operator to v0.5.4: Rolls up four upstream controller bug fixes: the operator no longer exempts the bootstrap seed from crash-loop self-heal, self-heal now also covers memory-backed etcd members, --initial-cluster-state is derived from cluster phase instead of the seed, and each EtcdCluster's PodDisruptionBudget switches from maxUnavailable to minAvailable (existing clusters are reconciled onto the new field automatically on upgrade). Tenant etcd clusters recover more reliably from member crashes and are less likely to have their PDB block a node drain (@androndo in #3529, backport #3538).

• fix(kubernetes): render the talos-reconcile Job for the default md0 group: On a managed Kubernetes cluster left with the default (undeclared) nodeGroups, the Job that creates each worker's TalosConfigTemplate and patches the control-plane's cert SANs only iterated the user-supplied node-group map, so it silently skipped the implicit md0 group. Nothing failed at install time, since md0 defaults to zero replicas, but the first scale-up (for example, autoscaler-driven growth after enabling ingress-nginx) left new Machines permanently blocked with no matching TalosConfigTemplate. The Job now iterates the same helper that produces the MachineDeployment, so md0 gets its reconcile Job like any explicitly declared group (@myasnikovdaniil in #3535, backport #3536).

• fix(keycloak-configure): patch HelmRelease in release namespace on teardown: The keycloak-configure pre-delete Job cleared the Flux HelmRelease finalizer in a hardcoded namespace that did not match where the release actually installs (cozy-keycloak), so its ServiceAccount was forbidden to patch it, the Job retried forever, and the HelmRelease stuck in Terminating — blocking any uninstall or reinstall of Keycloak. The teardown Job now templates both the release name and namespace from the Helm release itself, so teardown completes correctly (@lexfrei in #3372, backport #3478).

• fix(cozystack-basics): gate the hostname VAP policies on the VAP API: The hostname ValidatingAdmissionPolicy templates rendered unconditionally, so a first install on a cluster where the ValidatingAdmissionPolicy API is unavailable dropped the policies permanently — a later cluster upgrade that gains the API would not bring them back. The templates are now gated on .Capabilities.APIVersions.Has, so they render only where the API exists and are picked up automatically once it becomes available (@lexfrei in #3409, backport #3442).

• fix(seaweedfs): make naming audit fail closed on kubectl and payload errors: hack/seaweedfs-naming-audit.sh, used by operators to classify SeaweedFS instances before the naming-migration cleanup, was fail-open — any kubectl failure or unreadable Helm release payload produced an empty result table indistinguishable from a genuinely clean fleet. Since the runbook uses this script as the gate before deleting old PVCs, a transient API error could have green-lit destroying live data. Every query now fails loudly on error instead of silently reporting "nothing found," and incomplete evidence now falls back to a conservative "direction cannot be established" verdict rather than a wrong deletion candidate (@myasnikovdaniil in #3436, backport #3474).

• chore(release): don't activate kubernetes-nodes on the release-1.6 line: The kubernetes-nodes app package was not ready to ship on the 1.6 line, so its single include is removed from the iaas platform bundle — the platform stops activating the package on this line while its code, API types, and kubernetes-nodes-rd system package are all kept intact for when it is ready (@myasnikovdaniil in #3437).

Development, Testing, and CI/CD

• ci(release): carry the finalize fixes onto the 1.6 line: release-1.6 was cut before three release-pipeline fixes landed on main: dropping persist-credentials on checkout (which had let a stale GITHUB_TOKEN silently win over the app token, so the stable tag push created no workflow run and the automated changelog/docs backstops never fired, as happened for v1.6.0), publishing the GitHub release with the merged changelog as its body instead of a placeholder, and dropping a paths-ignore filter that could drop a changelog-only promotion PR. All three are backported so v1.6.1 releases correctly with real release notes (@myasnikovdaniil in #3530).

• fix(ci): overlay images from the PR base branch, and publish per-line artifacts: PR validation always overlaid unbuilt packages from cozystack-packages:main, so a release-1.6 PR was tested against main's controller binaries against its own line's charts — which is exactly what made #3437 above fail install deterministically with a schema-validation error. Each maintained release-X.Y branch now builds and publishes its own packages artifact, and PR validation overlays from the artifact matching the PR's own base branch (@myasnikovdaniil in #3471, backport #3514).

• fix(release): make promote-retag digest verification media-type-agnostic: The v1.6.0 finalize run aborted partway through promoting rc images to stable tags because its post-copy digest check used skopeo inspect --format '{{.Digest}}', which prints nothing for OCI artifacts like cozystack-packages, leaving most repositories without a stable tag and skipping the installer publish. The digest is now computed as the sha256 of the raw manifest, which works identically for container images and OCI artifacts, so promotion can no longer abort mid-way on this class of artifact (@myasnikovdaniil in #3435, backport #3473).

Other repositories

talm v0.34.0

• [talm] feat(charts): add preset value knobs: Exposes timeServers, control-plane component extraArgs, registryMirrors, per-host registryTLS, multiple Layer2 vips, network.preserveExisting, and network.extraLinks (bonds, VLANs, extra addresses and routes) as values on the cozystack, generic, and talm presets, so a node's Talos machine config can be described declaratively instead of via a template fork. Every knob defaults empty and a stock render stays byte-identical; each input Talos would reject fails fast at render time with a hinted error (@lexfrei in cozystack/talm#232).

• [talm] chore(deps): migrate to Helm 4 and drop the cozystack/talos fork: Moves talm's vendored Helm template engine from v3 to v4 and drops the cozystack/talos fork (carried solely for a --skip-verify flag, now reimplemented locally), tracking stock upstream Talos v1.13.7. Golden render snapshots confirm the generated machine config is unchanged for users (@lexfrei in cozystack/talm#231).

Documentation

• [website] feat(blog): add Cozystack 1.6 release and Blockstor announcement: Publishes the v1.6.0 release-announcement blog post — covering Talos Linux tenant workers, tenant-controlled OIDC, the SecurityGroup API, hierarchical quotas, and in-place etcd-operator adoption — alongside a companion post announcing the open-sourcing of Blockstor, the LINSTOR-compatible storage control plane (@tym83 in cozystack/website#641).

• [website] chore(blog): repair front matter, links and bundle names: Follow-up cleanup on the two new blog posts, fixing front matter, internal links, and page bundle names (@tym83 in cozystack/website@4b5d1ad).

• [website] chore: upgrade Hugo to 0.164.0 and convert HTML content to markdown: Upgrades the site generator to Hugo 0.164.0, requiring Node 22 under its new node permission model, and converts remaining raw-HTML content to markdown along the way, keeping the site buildable on current tooling (@tym83 in cozystack/website#636).

• [website] chore(blog): make the two taxonomy axes disjoint: Cleans up the blog's tagging so its two taxonomy axes no longer overlap, making blog post categorization and filtering more consistent (@tym83 in cozystack/website#635).

• [website] docs(talm): describe the preset value knobs for network and registries: Documents the new talm preset value knobs (cozystack/talm#232) for time servers, control-plane extra args, registry mirrors and TLS, VIPs, and network links, so operators can find the declarative equivalents to a template fork (@lexfrei in cozystack/website#633).

• [website] feat(hack): generate docs from a pre-tag ref and fail loudly on fetch errors: Hardens the docs-generation tooling to build from a ...

Cozystack v1.6.0

Cozystack v1.6.0 replaces the Ubuntu + kubeadm bootstrap of tenant Kubernetes workers with Talos Linux driven by Cluster API, completes the etcd-operator v1alpha2 migration with in-place adoption of live clusters, adds OIDC single sign-on for tenant kube-apiservers and per-instance Grafana, introduces the tenant-facing SecurityGroup firewall API, makes tenant resource quotas hierarchical so a sub-tenant can no longer escalate past its parent's budget, gives Keycloak an optional KMS-encrypting database proxy, a separate admin hostname and S3 backups, propagates operator-provided wildcard certificates down to every tenant termination point, and reworks the release pipeline around immutable tags and rc→stable promotion so a stable release is the byte-identical promotion of the release candidate that was tested. A ten-PR sweep also makes application deletion actually reclaim its storage instead of leaking PVCs. The release rolls up every fix from v1.5.1, v1.5.2 and v1.5.3.

Platform components bumped in this release: Talos v1.13.0 → v1.13.6 (KVM guest-escape CVE fixes), etcd-operator v0.4.5 → v0.5.2 (new etcd-operator.cozystack.io/v1alpha2 API), Cilium 1.19.3 → 1.19.5, KubeVirt v1.8.4, Velero 1.17.0 → 1.18.1, Vertical Pod Autoscaler 1.3.0 → 1.5.0, Harbor 2.14.2 → 2.15.1, Keycloak 26.5.2 → 26.6.3, LINSTOR 1.33.2 → 1.33.3 with linstor-csi v1.11.2, FoundationDB operator v2.13.0 → v2.30.0, HAMi 2.8.1 → 2.9.0, Percona MongoDB operator 1.21.1 → 1.22.0, OpenBao v2.5.0 → v2.5.1, CoreDNS chart 1.43.2 → 1.46.0, csi-driver-nfs 4.11.0 → 4.13.3, OpenCost 1.111.0 → 1.120.3, NATS 2.11.8 → 2.11.10, linstor-scheduler v0.3.2 → v0.3.6, and ouroboros 0.8.0 → 0.8.1.

Note: Items marked (shipped in v1.5.x) were also released in the v1.5.1, v1.5.2 or v1.5.3 patch releases.

⚠️ Breaking Changes and Required Actions

This is the largest upgrade surface since v1.0. Read this section in full before upgrading — three of the items below can block or wedge the upgrade if their preconditions are not met.

The platform migration targetVersion moves from 45 (v1.5.x) to 54, which means migrations 45 through 53 run as pre-upgrade hooks on the way to v1.6.0.

Pre-upgrade checks

Run these against the management cluster before applying the v1.6.0 Platform Package.

1. etcd adoption needs a reachable backup target (migration 50 — hard-fails the upgrade).

Migration 50 adopts every legacy etcd.aenix.io/v1alpha1 cluster onto the new v1alpha2 operator and takes a mandatory pre-adoption etcd snapshot first. If it cannot resolve the platform-managed snapshot target it exits 1 and halts the upgrade (#2859, #3270, #3335).

# Do you have legacy etcd clusters at all? If this prints nothing, migration 50 is a no-op.
kubectl get etcdclusters.etcd.aenix.io -A

# The snapshot target: the cozy-default-etcd strategy and the projected credentials Secret.
kubectl get etcds.strategy.backups.cozystack.io cozy-default-etcd
kubectl get secret cozy-backups-creds -n cozy-velero \
  -o jsonpath='{.data.AWS_ACCESS_KEY_ID}{"\n"}'

# The bucket behind it must be healthy.
kubectl get buckets.apps.cozystack.io cozy-backups -n tenant-root

If the Secret or the strategy is missing and you have legacy etcd clusters, either repair the cozy-backups Bucket / backupstrategy-controller first, or accept adopting live etcd without a snapshot by taking the documented escape hatch (see Manual actions required).

2. SeaweedFS naming audit (chart refuses to render for two classes of tenant).

The v1.5.0 SeaweedFS chart bump to 4.31 renamed workloads from chart-based (seaweedfs-*) to release-based (seaweedfs-system-*) names. v1.6.0 pins fullnameOverride: seaweedfs and adopts the running set in place, but two states cannot be adopted automatically and the chart fails the render rather than guess (#3282, #3339).

# Step 0 — seaweedfs-db ownership. A missing row means the filer metadata is ALREADY lost.
kubectl get cluster.postgresql.cnpg.io -A \
  -o custom-columns='NS:.metadata.namespace,NAME:.metadata.name,OWNER:.metadata.annotations.meta\.helm\.sh/release-name,KEEP:.metadata.annotations.helm\.sh/resource-policy'

# Step 1 — classify every SeaweedFS instance (read-only).
hack/seaweedfs-naming-audit.sh

Correction, added after release: the copy of that script shipped in v1.6.0 fails open, so a clean result from it is not evidence. Every kubectl call in the v1.6.0 version is silenced with 2>/dev/null, so a timeout, an RBAC denial or any transient API error produces an empty "all clean" table byte-identical to an honestly clean fleet — on the one script whose output gates a runbook step that deletes PVCs. Fixed in #3436 and backported as #3474, both merged 2026-07-28, so v1.6.1 and later carry a version that prints FATAL and exits non-zero instead. Run the audit from a v1.6.1-or-later checkout, read the exit code rather than the table, and re-run it if the only run you have was against the v1.6.0 copy.

Class L needs no action. Class S (installed fresh on 1.5.x) and class MIXED (both naming generations present) must be recovered before upgrading — follow docs/operations/seaweedfs-431-rename-recovery.md. A cluster that went 1.4.x → 1.6.0 directly never renamed and is unaffected.

3. Tenant Kubernetes clusters still on v1.30.

v1.30 leaves the Talos↔Kubernetes support matrix and the chart now refuses to render it. Migration 46 patches live CRs from v1.30 to v1.31, but a GitOps-managed CR is re-overwritten by the next source reconcile (#2931).

kubectl get kuberneteses.apps.cozystack.io -A \
  -o custom-columns='NS:.metadata.namespace,NAME:.metadata.name,VERSION:.spec.version'

Any row showing v1.30 that is reconciled from Git must have spec.version bumped to v1.31 (or newer) in Git, before or together with the platform upgrade.

4. Hand-made tenant StorageClasses that collide with propagated LINSTOR classes.

Remote-accessible LINSTOR StorageClasses are now created inside each tenant cluster under the same name. A manually created tenant StorageClass with a colliding name (typically replicated) blocks the propagated class and stalls the tenant CSI release (#2872).

# Inside each tenant cluster:
kubectl get storageclass replicated -o jsonpath='{.metadata.annotations.meta\.helm\.sh/release-name}{"\n"}'

Delete any such class that is not Helm-managed before upgrading. Infra classes that must stay node-local need an explicit linstor.csi.linbit.com/allowRemoteVolumeAccess: "false" — an absent annotation is treated as remote-accessible.

5. Deprecated etcd tenant-module backup values.

The backup.* block on the etcd tenant module is removed in this release. Confirm nothing still relies on it (#2859).

kubectl get etcds.apps.cozystack.io -A -o jsonpath='{range .items[*]}{.metadata.namespace}/{.metadata.name}{"\t"}{.spec.backup.enabled}{"\n"}{end}'

6. If you are coming from v1.4.x, the v1.5.0 requirement still applies: the management cluster (and any tenant cluster enabling the Flux addon) must be on Kubernetes 1.33+.

Breaking changes

• Tenant Kubernetes workers move from Ubuntu + kubeadm to Talos Linux. What changed: worker bootstrap is now driven by cluster-api-bootstrap-provider-talos (CABPT) with a talos-csr-signer sidecar in the Kamaji control-plane pod; workers boot the Talos openstack image streamed in by CDI. Who is affected: every existing tenant Kubernetes cluster. What to do: nothing mandatory — existing machines roll over to Talos workers automatically on the first reconcile after upgrade — but plan for a full worker-pool replacement per tenant cluster, and expect worker disks to be reprovisioned and container images re-pulled. The separate disk-kubelet PVC is gone; nodeGroups[*].diskSize now sizes the single Talos system disk (@myasnikovdaniil in #2931).

• Kubernetes v1.30 removed from the tenant support matrix. What changed: the version enum drops v1.30; the chart fails the render with an explicit guard. Who is affected: tenants still pinned to v1.30. What to do: migration 46 bumps live CRs to v1.31; GitOps-managed CRs must be bumped in Git (see pre-upgrade check 3) (#2931).

• Worker MachineHealthCheck remediation is now ON by default. What changed: the worker MHC maxUnhealthy moved from a hard-coded 0 (remediation effectively disabled) to nodeHealthCheck.maxUnhealthy, defaulting to "50%" — CAPI now deletes and replaces unhealthy worker Machines. Who is affected: every tenant Kubernetes cluster. What to do: set nodeHealthCheck.maxUnhealthy: "0%" to keep the previous behaviour until your fleet is stable on Talos workers (#2931).

• The default md0 node group is no longer merged into every cluster. What changed: nodeGroups defaults to {} and the built-in md0 is applied only when no node groups are configured, so it can finally be removed or replaced. Who is affected: clusters that defined their own node groups and were silently also getting md0. What to do: nothing — migration 47 pins md0 explicitly on existing CRs to preserve the live topology; remove it afterwards if you do not want it. Migration 47 is fail-closed: a read or patch failure aborts the upgrade rather than let Helm prune a live md0 MachineDeployment (@myasnikovdaniil in #2936).

• Fresh tenant clusters with nodeGroups: {} come up with zero workers. What changed: the chart no longer manages MachineDeployment.spec.replicas — the cluster-autoscaler owns it alone, seeded from minReplicas: 0. *Who is affect...

v1.5.2 (2026-07-03)

A patch release with ten bug fixes covering managed Kubernetes, MariaDB, Kafka, Harbor, SeaweedFS, VictoriaMetrics-operator dependency gating, Kamaji, and platform install ordering, plus new documentation and a talm update adding encrypted user values.

Fixes

• fix(kamaji): backport datastore unused-deletion deadlock fix: Deleting a Tenant whose etcd DataStore had no more referencing TenantControlPlanes left the DataStore stuck Terminating (blocked by the kamaji.clastix.io/TenantControlPlane finalizer), hanging the etcd Helm uninstall and wedging the tenant namespace. Backports the upstream fix (clastix/kamaji#1122, fixing clastix/kamaji#1115) as a source patch on the pinned kamaji build, so deleting the last control plane on a DataStore now cleans up correctly (@myasnikovdaniil in #3146, backport #3187).

• fix(kubernetes): gate vmop cert-manager-crds dependency on certManager.enabled: The victoria-metrics-operator HelmRelease listed cert-manager-crds in dependsOn unconditionally, even though that HelmRelease is only created when certManager.enabled is true. Clusters running monitoringAgents.enabled=true with certManager.enabled=false had victoria-metrics-operator permanently blocked on a dependency that would never exist, cascading to monitoring-agents and vertical-pod-autoscaler never becoming ready. The dependency is now gated on the same condition that creates it (@myasnikovdaniil in #3130, backport #3185).

• fix(tenant): gate VM-resource HelmReleases on victoria-metrics-operator readiness: The tenant etcd, ingress, and monitoring HelmReleases create VictoriaMetrics-operator custom resources guarded by a validating webhook with failurePolicy: Fail. On a cold install the operator pod could be briefly unavailable, so these releases would reconcile before the webhook was serving and fail with "connection refused" — the failed-install/rollback/retry churn could then outlast the install-readiness deadline and surface as a spurious failure. These releases now dependsOn the victoria-metrics-operator HelmRelease so they wait for it to become Ready first, eliminating the race on fresh installs (@lexfrei in #3141, backport #3166).

• fix(mariadb): re-guard replication so single-replica passes the operator webhook: A prior change made replication.enabled: true unconditional so the operator would always create -primary/-secondary services, but the mariadb-operator validating webhook rejects replication when spec.replicas == 1, so single-replica MariaDB instances could no longer be created — the HelmRelease never became Ready. Replication is now re-guarded on replicas > 1; single-replica installs keep external access via a LoadBalancer service on the bare release name, and backups, RBAC, and dashboards are updated to recognize both topologies (@scooby87 in #3180, backport #3183).

• fix(kubernetes): make node group resources optional when instanceType is set: A managed-Kubernetes node group sized purely by instanceType (with resources omitted) failed values.schema.json validation because resources was required on every node group. resources is now optional: a node group sized purely by instanceType passes validation and installs, while a node group setting both instanceType and explicit resources has the explicit resources take precedence, so KubeVirt does not reject a VM referencing an instancetype while overriding domain.cpu/memory (@lexfrei in #3121, backport #3156).

• fix(seaweedfs): enforce read-only bucket access by bumping cosi-driver to v0.3.1: The bundled SeaweedFS COSI driver (v0.1.2) hardcoded read-write S3 permissions for every BucketAccess and ignored the chart's -readonly BucketAccessClass, so credentials meant to be read-only could still upload and delete objects. Bumping the driver to v0.3.1 makes it honor accessPolicy: readonly and issue Read/List-only credentials, closing the privilege gap for tenants relying on read-only bucket access (@lexfrei in #3136, backport #3153).

• fix(platform): order backupstrategy-controller after cozystack-basics: backupstrategy-controller creates the cozy-backups Bucket in the tenant-root namespace, which cozystack-basics provisions. On a fresh install the Bucket could race namespace creation and fail with namespaces "tenant-root" not found, which then raced the install-wait deadline and failed unrelated fresh installs. A dependsOn edge on cozystack-basics now guarantees the target namespace exists first (@lexfrei in #3037, backport #3079).

• fix(kafka): set entity-operator resources to avoid OOM under tenant LimitRange: Kafka's entityOperator topic-operator and user-operator containers carried no resource requests/limits, so in tenant namespaces — where the default LimitRange caps containers at 128Mi — the JVM processes were OOMKilled on startup. The entity-operator stayed in CrashLoopBackOff, KafkaTopic/KafkaUser objects never reconciled, and topics could get stuck on the strimzi.io/topic-operator finalizer during teardown. Explicit 256Mi/512Mi requests/limits are now set so the operators no longer inherit the namespace default (@myasnikovdaniil in #2934, backport #3054).

• fix(harbor): preserve jobservice and trivy storageClass on upgrade: The cozy-harbor wrapper never forwarded storageClass to the jobservice jobLog PVC, so it was created without a storageClassName and picked up the cluster default via default-StorageClass admission — and because PVC storageClassName is immutable, this produced rendered-vs-live drift on every upgrade. The Trivy StatefulSet's volume claim template had the same latent issue. Both now preserve an existing PVC's class via lookup and fall back to the configured storageClass for fresh installs (@myasnikovdaniil in #2930, backport #3147).

• fix(seaweedfs): point S3 consumers at the reachable S3 service: Renaming the SeaweedFS S3 Service to <name>-s3 left several consumers — the S3 ingress backend, the iceberg ingress backend, and the in-cluster COSI provisioner endpoint — still resolving the old <fullname>-s3 name or the wrong port, so the external S3 endpoint returned 503 via ingress-nginx and in-cluster COSI paths defaulted to port 443 (or the filer service, which serves no S3) instead of the real S3 service on port 8333. All consumers now resolve seaweedfs-s3 on the correct port (@lexfrei in #3119, backport #3127).

Other repositories

talm v0.32.0

• [talm] feat: first-class encrypted user values, honored at template and apply: Adds support for encrypted user secret values (a registry password, a KMS plugin's secret-id, etc.) that chart templates consume — encrypted at rest in git with the existing talm.key, decrypted in memory only at render time, and kept out of committed node files and terminal output. A value file named *.encrypted.yaml referenced from templateOptions.valueFiles is age-decrypted in memory at both talm template and talm apply; talm init --encrypt/--decrypt covers values-secret.yaml ↔ values-secret.encrypted.yaml. talm apply now also honors the full value-source flag set and Chart.yaml's templateOptions, and secret-bearing fields are redacted by default from rendered node files, stdout, and the apply drift preview, with --show-secrets/--show-secrets-in-drift to reveal them on demand (@lexfrei in cozystack/talm#222).

Documentation

• [website] docs(operations): add containerized GPU workloads guide: Publishes a new operations guide for the container variant of cozystack.gpu-operator, covering containerized GPU workloads (CUDA pods, ML training, inference) on GPU nodes that already ship the NVIDIA driver and nvidia-container-toolkit via the distro package manager (@lexfrei in cozystack/website#555).

• [website] docs(talos): warn against changing op: on the machine files block: Adds a warning next to the Talos machine.files block in the bare-metal bootstrap docs, explaining that Talos rejects op: create for files outside /var, so the /etc/lvm/lvm.conf entry must keep op: overwrite — changing it silently fails the WriteUserFiles boot step and reboot-loops the node with only the opaque bootstrap is not available yet error (@lexfrei in cozystack/website#575).

• [website] docs(talm): document chart drift detection and strict mode: Documents the chart drift detection shipping with talm (cozystack/talm#216) — the library-content and preset-baseline drift warnings, the .talm-preset.lock file and the advice to commit it, talm init --update re-sync semantics, and the strictCharts: true/--strict-charts escalation with its unverifiable-baseline behavior (@lexfrei in cozystack/website#577).

• [website] docs(platform): document publishing.certificates wildcard options: Documents two previously-undocumented publishing.certificates.* platform-package options, including the opt-in wildcard mode that issues one shared *.<root-host> certificate via a DNS-01 solver instead of a per-host ACME certificate, helping operators avoid Let's Encrypt rate limits at scale (@lexfrei in cozystack/website#588).

• [website] fix(docs): correct stale version pins and prevent next.yaml drift: The version-pin data file for the v1.5 docs was a stale copy of `n...