CKS: add an editable per-cluster management-access CIDR policy #14212

Relationship to existing issue

Follow-up to #13970.

#13970 correctly tracks the hard-coded 0.0.0.0/0 source used by CKS for node SSH and also identifies the Kubernetes API and VPC ACL variants. This follow-up defines the cluster-level API/UI contract, lifecycle reconciliation, network-mode behavior, and rule ownership needed to solve the problem consistently rather than only replacing one constant.

Problem

CKS-managed ingress to cluster management endpoints cannot be configured per cluster:

• node SSH firewall rules are created with 0.0.0.0/0;
• the TCP/6443 Kubernetes API firewall path is also unrestricted;
• VPC ACL rules are created without an explicit cidrlist, so the API default is used;
• SG-enabled zones use one account-wide CKSSecurityGroup-<account UUID> with unrestricted SSH and 6443 ingress;
• create/list APIs and the UI have no cluster property representing the desired policy.

Manual narrowing is not a supported workaround. KubernetesClusterScaleWorker.scaleKubernetesClusterIsolatedNetworkRules() calls removeSshFirewallRule(), which identifies a rule mainly by its public port / related PF destination. It then calls setupKubernetesClusterIsolatedNetworkRules(), which recreates the rule with 0.0.0.0/0. Node-add follows the same unrestricted provisioning path. Since an existing firewall rule's CIDR list is immutable, the fix needs a controlled rule replacement workflow.

The current removal helpers also risk selecting unrelated user-managed rules that happen to use the same public IP and ports.

Observed on current main at analysis time: 602d9ec3e03e6350b55dd1349118ea2ccb766cca.

Proposed feature

Add an editable Cloud-managed CKS cluster property:

managementaccesscidrlist=10.20.0.0/16,192.0.2.40/32,2001:db8:1200::/48

In the first version, the same list controls source access to:

• node SSH, including the public SSH port-forward range;
• the Kubernetes API on TCP/6443.

Required API/UI behavior:

• add managementaccesscidrlist to createKubernetesCluster;
• add an async updateKubernetesClusterAccess API;
• return the canonical list and its enforcement mode in KubernetesClusterResponse;
• add Management access networks to the create form and an edit action in cluster details;
• validate IPv4/IPv6 CIDRs, trim, deduplicate, and reject malformed or empty lists;
• show an explicit warning for 0.0.0.0/0 and ::/0;
• warn that the policy must permit the source address used by the Management Server for CKS SSH lifecycle operations.

Add an inherited account-scoped CSV configuration such as:

cloud.kubernetes.cluster.management.access.cidrs

It should default to 0.0.0.0/0 for compatibility. When the create parameter is omitted, resolve the owner account's inherited account/domain/global value and snapshot it on the new cluster. Later configuration changes should affect new clusters only.

Existing clusters without a stored value should retain the legacy effective value 0.0.0.0/0 until explicitly updated.

Network-specific behavior

Isolated network

Apply the list to CKS-owned ingress firewall rules on the cluster public/source-NAT IP for the SSH public ports and TCP/6443. Port-forwarding and load-balancer rules remain translation/backend objects; the firewall is the source-policy layer.

VPC isolated tier

Apply the list to the CKS-created ingress ACL entries for TCP/22 and TCP/6443.

A restricted policy cannot be enforced with the immutable default-allow ACL. In that case, reject create/update with a clear request to use a custom ACL. Preserve the existing default-deny validation.

SG-enabled zone

Use a deterministic per-cluster security group with ingress TCP/22 and TCP/6443 from the configured CIDRs. Do not edit the current account-wide group for one cluster, because that would change every CKS cluster in the account.

Existing clusters using the account-wide group need a safe stopped-cluster migration to a per-cluster group, or a clear rejection if the VM security-group associations cannot be changed safely.

This branch is conditional on the zone actually being security-group-enabled; a shared network must not be assumed to have security-group enforcement merely because it is shared.

Shared/routed direct-access network without enforcement

If CKS has no SG, firewall, or ACL layer that can enforce the list, report enforcement as external and reject a restricted create/update request. Do not claim that a stored CIDR list is enforced.

Lifecycle and ownership requirements

Persist the desired policy as cluster state (the existing kubernetes_cluster_details table is sufficient for the list). Do not infer desired CIDRs from currently installed rules.

Track CKS-created firewall, ACL, PF, and LB resources by explicit cluster ownership/identity. Port-only matching is not ownership.

Create/update and every lifecycle path must use the stored list:

• create and start;
• stop/start;
• scale up/down;
• add/remove node;
• upgrade when rules are recreated;
• delete cleanup.

CIDR update must be idempotent and retryable. Because firewall/ACL CIDR lists are immutable, it should replace only the old CKS-owned rules, verify the new rules, then persist the new desired policy. A failed update must not silently leave the persisted policy different from the effective network rules.

Acceptance criteria

1. A user can set one or more CIDRs when creating a Cloud-managed CKS cluster.
2. An authorized user can edit the list later via API and UI.
3. List/get APIs return the canonical list and enforcement mode (FIREWALL, VPC_ACL, SECURITY_GROUP, or EXTERNAL).
4. Isolated-network SSH and API firewall rules use the exact stored list.
5. Custom VPC ACL entries for SSH and API use the exact stored list.
6. New SG-enabled clusters in the same account can have different policies.
7. Restricted policies are rejected when the selected network cannot enforce them.
8. Scale and add/remove-node preserve the exact list and never restore 0.0.0.0/0.
9. Unrelated user-created rules on the same ports are never revoked by CKS.
10. Existing clusters remain backward compatible until explicitly updated.
11. Unit/integration tests cover isolated, VPC, SG-enabled, and unenforceable direct-access modes.

Scope clarification

This controls only management ingress created by CKS for SSH and the Kubernetes API. It does not replace Service.spec.loadBalancerSourceRanges or the CloudStack Kubernetes provider's source-CIDR annotation for arbitrary Kubernetes Service objects.