fix(kubernetes): render the talos-reconcile Job for the default md0 group - #3535
Conversation
…roup The Job in templates/talos/talos-reconcile-job.yaml is the only producer of the TalosConfigTemplate that each worker MachineDeployment names in spec.template.spec.bootstrap.configRef; cluster.yaml renders the reference but deliberately not the object. Its loop read .Values.nodeGroups directly, while the MachineDeployment loop reads the effective set through the kubernetes.nodeGroups helper. For a cluster that supplies no nodeGroups the two disagree: the helper's else-branch emits the built-in md0 group, so the md0 MachineDeployment renders, but the raw map is empty and no Job renders at all. Every Machine the cluster-autoscaler adds to md0 then blocks indefinitely on a TalosConfigTemplate that nothing will ever create, and the KamajiControlPlane spec.network.certSANs patch the same Job performs is skipped too. Range over the helper so the Job set tracks the MachineDeployment set exactly. No change for a cluster that declares its own groups, and md0 stays removable: the helper's if-branch keeps a user-supplied map authoritative. The gap survived because every helm-unittest fixture and e2e suite declares md0 explicitly. tests/nodegroups_default_test.yaml does render the empty-nodeGroups case but lists only templates/cluster.yaml, so it never looked at the Job. The new suite pins both halves of the helper contract and fails without this fix. Fixes #3504 Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe Talos reconcile Job loop now uses effective node groups. Regression tests cover the built-in ChangesTalos reconcile node-group rendering
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related issues
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Aleksei Sviridkin (lexfrei)
left a comment
There was a problem hiding this comment.
LGTM. The fix is at the root cause and I could not break it.
Business context: a tenant cluster created with the default empty nodeGroups renders the md0 MachineDeployment but no talos-reconcile Job, so the TalosConfigTemplate the deployment references is never created and every autoscaler-added worker blocks on bootstrap; this makes the Job loop read the same effective group set as the MachineDeployment loop.
What I checked beyond reading the diff. The new suite fails on the merge-base template exactly as described: case 1 dies on the document count, case 2 passes. Reverting the loop to the raw map and, separately, making the helper merge md0 unconditionally both turn the suite red, so it guards the behaviour rather than restating it. Full chart suite is green at 191 tests, and the untouched content-hash fixtures in talos_templates_test.yaml confirm the explicit-groups render is byte-identical, so no existing cluster's Job name rotates. After this change no raw .Values.nodeGroups iteration remains in the chart, and the sibling kubernetes-nodes chart is single-pool with no map loop, so this bug class does not exist there.
|
Successfully created backport PR for |
Bring the worker-pool split branch up to date with main (111 commits), picking up the node-join and CI fixes it was missing: the 18m tenant node-join deadline, guest-fsync-off on ephemeral CI disks (#3455), kubevirt-csi client rate-limit raise (#3428), the Cilium ingress-IP race guard (#3430), and etcd-operator v0.5.4. Conflict resolution: main's #3535 (render the talos-reconcile Job for the default md0 group) modified packages/apps/kubernetes/templates/talos/ talos-reconcile-job.yaml, which this branch deletes because the worker split moves that Job into the kubernetes-nodes chart. Kept the parent template deleted and dropped the parent-scoped test talos_reconcile_nodegroups_test.yaml: the empty-nodeGroups md0-default gap it guards cannot occur in the per-pool child chart, and migration 54 materialises the implicit md0 into an explicit child HelmRelease. targetVersion resolves to 55 (migration 54 on top of main's 53). Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
What this PR does
Fixes #3504.
The chart has two loops over worker node groups and they read different sources.
templates/cluster.yamliterates the effective set through thekubernetes.nodeGroupshelper, whose else-branch supplies the built-inmd0group when the user declares none — that is how #2936 mademd0removable without a Helm values merge re-adding it.templates/talos/talos-reconcile-job.yamliterated the raw.Values.nodeGroupsmap instead, which is empty in exactly that case.The consequence is a dangling reference. The chart cannot render the
TalosConfigTemplateitself — it needs the apiserver Service ClusterIP, the Talos CA and the Kubernetes CA, none of which exist while Helm is executing the template — socluster.yamldeliberately stopped rendering it and the Job became its only producer, while theMachineDeploymentstill names it inspec.template.spec.bootstrap.configRef. On a cluster with the defaultnodeGroups,MachineDeployment/<release>-md0renders and zero Jobs render, soTalosConfigTemplate/<release>-md0is never created and CAPI blocks every Machine that ever joins the group. The same Job also patchesKamajiControlPlane.spec.network.certSANswith the live Service ClusterIP, so that is skipped too.Nothing fails at install time, because the built-in group carries
minReplicas: 0. The failure waits for the first scale-up, which is the documented path rather than a corner:values.yamltells operators that enabling the ingress-nginx addon on a cluster with defaultnodeGroupsmeans waiting for the cluster-autoscaler to bringmd0up in response to the controller Pods becoming Pending, and the autoscaler is deployed unconditionally whenever the tenant has an etcd DataStore.The fix is to range over the helper so the Job set tracks the MachineDeployment set exactly. A cluster that declares its own groups renders byte-identically — which is why the pinned content-hash fixtures in
tests/talos_templates_test.yamlare untouched — andmd0stays removable, because the fix defers to the helper rather than mergingmd0in unconditionally. On upgrade it is additive: the Job appears where there was none, applies theTalosConfigTemplatethe MachineDeployment already expects, and the content-hash name suffix means Helm creates a fresh Job rather than attempting to patch an immutable one.How this survived
Every helm-unittest fixture and every e2e suite declares
md0explicitly.tests/nodegroups_default_test.yamldoes render the empty-nodeGroupscase, but lists onlytemplates/cluster.yaml, so it never looked at a Job. The two OIDC chainsaw lanes do install with an empty map, but assert that the HelmRelease exists rather than that it becomes Ready, so a worker that never boots is invisible to them. Offlinehelm templatecannot reach this path at all:cluster.yaml's instanceType validator needs a livelookupforu1.medium.tests/talos_reconcile_nodegroups_test.yamlcloses the gap and pins both halves of the helper contract — an empty map yields exactly onemd0Job carryingGROUP_NAME=md0, and a two-group map yields exactly two Jobs with nomd0among them, so a future fix that mergesmd0in unconditionally fails here. Reverting the template change fails the first case on the document count, so it guards the behaviour rather than restating it.Screenshots
Not a UI change.
Downstream repositories
Walked the trigger map in
docs/agents/contributing.mdagainst the diff: it is one Helm template loop and one new helm-unittest suite insidepackages/apps/kubernetes. No package is added, renamed or removed; novalues.yaml,values.schema.json,ApplicationDefinition, version enum or default changes; nothing underhack/, no namespace, variant, label, annotation or metric renamed. The behaviour now matches whatvalues.yamlalready documents, so no reference page drifts.Release note
Summary by CodeRabbit
Bug Fixes
md0node group receives its Talos reconciliation Job when no node groups are configured.Tests