Skip to content

revert(ingress)!: remove broken publishing.exposure=loadBalancer mode - #2603

Merged
Andrei Kvapil (kvaps) merged 1 commit into
mainfrom
revert/ingress-loadbalancer-mode
May 19, 2026
Merged

revert(ingress)!: remove broken publishing.exposure=loadBalancer mode#2603
Andrei Kvapil (kvaps) merged 1 commit into
mainfrom
revert/ingress-loadbalancer-mode

Conversation

@kvaps

@kvaps Andrei Kvapil (kvaps) commented May 11, 2026

Copy link
Copy Markdown
Member

What this PR does

Reverts the publishing.exposure=loadBalancer mode for ingress-nginx added in #2468.

The mode rendered the ingress-nginx Service as type: LoadBalancer and a CiliumLoadBalancerIPPool over publishing.externalIPs, as a migration path off Service.spec.externalIPs (deprecated upstream in Kubernetes v1.36 per KEP-5707). In default Cozystack the implementation produces two failure modes:

  • publishing.externalIPs is the operator-supplied list of node public IPs (addresses the cluster nodes already hold and listen on, see docs section "4b. Node Public IP Setup"), not a virtual-VIP pool. Treating it as one and DNATing those IPs via Cilium LB-IPAM collides with the existing ingress-nginx DNAT on the same IPs (Service.spec.externalIPs path), so traffic is forwarded non-deterministically.
  • Cilium in default Cozystack runs with ipam.mode: kubernetes and no L2/BGP announcer, so a pool over addresses outside the node range is allocated but never advertised — the Service stays unreachable.

The KEP-5707 deprecation is real, but the right migration is a mechanism-agnostic tenant API knob translated by a Cozystack-side controller into whichever backend the cluster admin has configured (MetalLB pool, Cilium LB-IPAM, robotlb, externalIPs pin) — not a chart-side IPPool baked into the tenant chart. That follow-up belongs in its own PR.

Removed

  • publishing.exposure value (and the inline comment) from core/platform/values.yaml.
  • expose-mode plumbing line in core/platform/templates/apps.yaml.
  • CiliumLoadBalancerIPPool template at apps/tenant/templates/cilium-lb-pool.yaml.
  • type: LoadBalancer + externalTrafficPolicy: Local branch in extra/ingress/templates/nginx-ingress.yaml. The Service now renders as ClusterIP + spec.externalIPs when externalIPs are set, falling through to type: LoadBalancer when they are not — pre-feat(ingress): add loadBalancer exposure mode via CiliumLoadBalancerIPPool #2468 shape.
  • Exposure mode section from extra/ingress/README.md.
  • exposure_test.yaml fixtures in both charts.

Compatibility

Operators who explicitly set publishing.exposure=loadBalancer will see their ingress Service revert to ClusterIP + spec.externalIPs on upgrade. Anyone on the default value (externalIPs) is unaffected.

Release note

revert(ingress): remove the publishing.exposure=loadBalancer mode added in #2468. The default externalIPs path is unchanged; operators who explicitly opted into loadBalancer will have their ingress Service revert to ClusterIP + spec.externalIPs on upgrade. Migration off Service.spec.externalIPs (deprecated in KEP-5707) will be reintroduced via a mechanism-agnostic API knob in a follow-up.

Summary by CodeRabbit

Release Notes

  • Refactor

    • Removed conditional generation of the Cilium load‑balancer IP pool and simplified service exposure to rely on explicit external IP values.
    • Streamlined configuration for service exposure keys.
  • Documentation

    • Removed the exposure-mode guidance from ingress documentation.
  • Tests

    • Deleted the exposure-mode-related test suites covering various external IP and loadBalancer scenarios.

Review Change Stack

@coderabbitai

coderabbitai Bot commented May 11, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 71ee3717-6fa5-4192-8bc7-7c8b4f4a6416

📥 Commits

Reviewing files that changed from the base of the PR and between 0642ee0 and f919afe.

📒 Files selected for processing (7)
  • packages/apps/tenant/templates/cilium-lb-pool.yaml
  • packages/apps/tenant/tests/exposure_test.yaml
  • packages/core/platform/templates/apps.yaml
  • packages/core/platform/values.yaml
  • packages/extra/ingress/README.md
  • packages/extra/ingress/templates/nginx-ingress.yaml
  • packages/extra/ingress/tests/exposure_test.yaml
💤 Files with no reviewable changes (6)
  • packages/core/platform/templates/apps.yaml
  • packages/extra/ingress/tests/exposure_test.yaml
  • packages/apps/tenant/templates/cilium-lb-pool.yaml
  • packages/extra/ingress/README.md
  • packages/apps/tenant/tests/exposure_test.yaml
  • packages/core/platform/values.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/extra/ingress/templates/nginx-ingress.yaml

📝 Walkthrough

Walkthrough

This PR removes the expose-mode abstraction layer by replacing it with explicit configuration fields. The platform chart now generates separate expose-services, expose-ingress, and expose-external-ips values instead of deriving a single mode; ingress and tenant templates adapt accordingly by consuming these explicit fields directly and removing conditional mode-based logic.

Changes

Exposure Configuration Simplification

Layer / File(s) Summary
Configuration Schema Update
packages/core/platform/values.yaml
Removes publishing.exposure configuration block and its documentation.
Platform Template Value Generation
packages/core/platform/templates/apps.yaml
Generates three explicit exposure fields (expose-services, expose-ingress, expose-external-ips) in _cluster instead of a computed expose-mode.
Ingress Service Template Refactoring
packages/extra/ingress/templates/nginx-ingress.yaml
Uses _cluster["expose-external-ips"] directly; removes expose-mode parsing, validation logic, and mode-based conditional rendering.
Cilium Load Balancer Pool Feature Removal
packages/apps/tenant/templates/cilium-lb-pool.yaml
Removes the CiliumLoadBalancerIPPool resource template that depended on the loadBalancer exposure mode.
Documentation Updates
packages/extra/ingress/README.md
Removes the "Exposure mode" section documenting publishing.exposure behavior and rendering shapes.
Test Suite Cleanup
packages/apps/tenant/tests/exposure_test.yaml, packages/extra/ingress/tests/exposure_test.yaml
Removes tests validating the old exposure-mode behavior, including CIDR generation, mode validation, error handling, and namespace-dependent fallback cases.

🎯 3 (Moderate) | ⏱️ ~25 minutes

🐰 The exposure mode has now been slain,
Replaced by config fields so plain,
No more conditional maze,
Direct values blaze,
Templates simpler—less to maintain! 🌟

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'revert(ingress)!: remove broken publishing.exposure=loadBalancer mode' accurately describes the main change: reverting and removing a broken feature from the ingress configuration, matching the comprehensive scope of deletions across multiple files.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch revert/ingress-loadbalancer-mode

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added size/L This PR changes 100-499 lines, ignoring generated files area/networking Issues or PRs related to networking (ingress, gateway, vpn, metallb, kube-ovn) kind/breaking-change Indicates the change introduces a breaking API or behaviour change labels May 11, 2026
@lexfrei
Aleksei Sviridkin (lexfrei) marked this pull request as ready for review May 11, 2026 08:05
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request reverts the recently introduced 'publishing.exposure=loadBalancer' mode for ingress-nginx. The implementation was found to cause non-deterministic traffic routing and service unavailability due to conflicts with existing DNAT configurations and a lack of IP advertisement in the default Cozystack environment. By removing this mode, the system returns to its previous, stable state while allowing for a more robust, mechanism-agnostic migration strategy to be developed in the future.

Highlights

  • Reversion of LoadBalancer Mode: Removed the 'publishing.exposure=loadBalancer' mode for ingress-nginx, which was causing traffic routing issues and service reachability problems in the default Cozystack configuration.
  • Cleanup of Associated Resources: Deleted the 'CiliumLoadBalancerIPPool' template and associated test fixtures, and removed the 'expose-mode' configuration plumbing from platform and ingress charts.
  • Restoration of Default Behavior: The ingress-nginx Service now defaults back to 'ClusterIP' with 'spec.externalIPs' when configured, reverting to the stable behavior prior to the introduction of the loadBalancer mode.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment Gemini (@gemini-code-assist) Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@dosubot dosubot Bot added the kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API label May 11, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/extra/ingress/templates/nginx-ingress.yaml (1)

44-52: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Remove the remaining LoadBalancer fallback from ingress service rendering

Line 50 still sets type: LoadBalancer in the else branch, which effectively keeps the behavior this PR is reverting. This can still produce unreachable services in environments without L2/BGP announcement support. Prefer rendering ClusterIP by default and only adding externalIPs when configured.

Suggested patch
         service:
-          {{- if and (eq $exposeIngress .Release.Namespace) $exposeExternalIPs }}
+          type: ClusterIP
+          externalTrafficPolicy: Cluster
+          {{- if and (eq $exposeIngress .Release.Namespace) $exposeExternalIPs }}
           externalIPs:
             {{- toYaml (splitList "," $exposeExternalIPs) | nindent 12 }}
-          type: ClusterIP
-          externalTrafficPolicy: Cluster
-          {{- else }}
-          type: LoadBalancer
-          externalTrafficPolicy: Local
           {{- end }}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/extra/ingress/templates/nginx-ingress.yaml` around lines 44 - 52,
The template currently falls back to rendering type: LoadBalancer and
externalTrafficPolicy: Local in the else branch of the conditional that checks
and (eq $exposeIngress .Release.Namespace) $exposeExternalIPs; change this so
the default/else branch renders a ClusterIP service (no LoadBalancer) and do not
set externalTrafficPolicy there — only inject externalIPs and (if needed)
externalTrafficPolicy when $exposeExternalIPs is provided in the if branch;
remove the type: LoadBalancer and externalTrafficPolicy: Local lines from the
else branch and ensure type: ClusterIP is the default output unless externalIPs
are explicitly configured.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@packages/extra/ingress/templates/nginx-ingress.yaml`:
- Around line 44-52: The template currently falls back to rendering type:
LoadBalancer and externalTrafficPolicy: Local in the else branch of the
conditional that checks and (eq $exposeIngress .Release.Namespace)
$exposeExternalIPs; change this so the default/else branch renders a ClusterIP
service (no LoadBalancer) and do not set externalTrafficPolicy there — only
inject externalIPs and (if needed) externalTrafficPolicy when $exposeExternalIPs
is provided in the if branch; remove the type: LoadBalancer and
externalTrafficPolicy: Local lines from the else branch and ensure type:
ClusterIP is the default output unless externalIPs are explicitly configured.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 9a057e64-bd6d-4c51-9f7e-712f9dd88db0

📥 Commits

Reviewing files that changed from the base of the PR and between 99e7070 and 0642ee0.

📒 Files selected for processing (7)
  • packages/apps/tenant/templates/cilium-lb-pool.yaml
  • packages/apps/tenant/tests/exposure_test.yaml
  • packages/core/platform/templates/apps.yaml
  • packages/core/platform/values.yaml
  • packages/extra/ingress/README.md
  • packages/extra/ingress/templates/nginx-ingress.yaml
  • packages/extra/ingress/tests/exposure_test.yaml
💤 Files with no reviewable changes (6)
  • packages/extra/ingress/README.md
  • packages/apps/tenant/tests/exposure_test.yaml
  • packages/apps/tenant/templates/cilium-lb-pool.yaml
  • packages/core/platform/templates/apps.yaml
  • packages/core/platform/values.yaml
  • packages/extra/ingress/tests/exposure_test.yaml

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request reverts the "loadBalancer" exposure mode for ingress services, removing the CiliumLoadBalancerIPPool template, configuration options, and associated tests. Feedback highlights that the new implementation for externalIPs fails to filter empty strings from the IP list, potentially resulting in invalid Service manifests. Furthermore, the PR title uses an unauthorized commit type, violating the project's conventional commit guidelines.

Comment on lines +44 to +46
{{- if and (eq $exposeIngress .Release.Namespace) $exposeExternalIPs }}
externalIPs:
{{- toYaml $exposeIPsList | nindent 12 }}
{{- toYaml (splitList "," $exposeExternalIPs) | nindent 12 }}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The previous implementation (which was removed in this PR) safely filtered out empty strings from the IP list. The new implementation using splitList directly on the raw string can result in empty strings being passed to the externalIPs field (e.g., if there is a trailing comma), which is invalid in a Kubernetes Service. Using the compact function will restore the previous robustness by removing any empty elements from the list.

          {{- $ips := splitList "," $exposeExternalIPs | compact }}
          {{- if and (eq $exposeIngress .Release.Namespace) $ips }}
          externalIPs:
            {{- toYaml $ips | nindent 12 }}

@@ -1,19 +1,5 @@
{{- $exposeIngress := (index .Values._cluster "expose-ingress") | default "tenant-root" }}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

The pull request title uses the commit type revert, which is not included in the list of valid types specified in the repository style guide. Please use one of the allowed types: feat, fix, docs, style, refactor, perf, test, build, ci, chore.

References
  1. Valid commit types are feat, fix, docs, style, refactor, perf, test, build, ci, chore. (link)

Reverts the ingress loadBalancer exposure mode added in #2468.

The mode rendered the ingress-nginx Service as type: LoadBalancer and a
CiliumLoadBalancerIPPool over publishing.externalIPs, as a migration path
off Service.spec.externalIPs (deprecated upstream in Kubernetes v1.36 per
KEP-5707). In default Cozystack the implementation produces two failure
modes:

- publishing.externalIPs is the operator-supplied list of node public IPs
  (addresses the cluster nodes already hold and listen on), not a
  virtual-VIP pool. Treating it as one and DNATing those IPs via Cilium
  LB-IPAM collides with the existing ingress-nginx DNAT on the same IPs
  (Service.spec.externalIPs path), so traffic is forwarded
  non-deterministically.
- Cilium in default Cozystack runs with ipam.mode: kubernetes and no
  L2/BGP announcer, so a pool over addresses outside the node range is
  allocated but never advertised — the Service stays unreachable.

The KEP-5707 deprecation is real, but the right migration is a
mechanism-agnostic tenant API knob translated by a Cozystack-side
controller into whichever backend the cluster admin has configured
(MetalLB pool, Cilium LB-IPAM, robotlb, externalIPs pin) — not a
chart-side IPPool baked into the tenant chart. That follow-up belongs in
its own PR.

Removed:
- publishing.exposure value (and the inline comment) from
  core/platform/values.yaml
- expose-mode plumbing line in core/platform/templates/apps.yaml
- CiliumLoadBalancerIPPool template at
  apps/tenant/templates/cilium-lb-pool.yaml
- type: LoadBalancer + externalTrafficPolicy: Local branch in
  extra/ingress/templates/nginx-ingress.yaml (Service renders back as
  ClusterIP + spec.externalIPs when externalIPs are set, falling through
  to type: LoadBalancer when they are not — pre-#2468 shape)
- Exposure mode section from extra/ingress/README.md
- exposure_test.yaml fixtures in both charts

Operators who explicitly set publishing.exposure=loadBalancer will see
their ingress Service revert to ClusterIP + spec.externalIPs on upgrade.
Anyone on the default value (externalIPs) is unaffected.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
@kvaps
Andrei Kvapil (kvaps) force-pushed the revert/ingress-loadbalancer-mode branch from 0642ee0 to f919afe Compare May 11, 2026 08:42
@kvaps
Andrei Kvapil (kvaps) merged commit e846777 into main May 19, 2026
15 of 18 checks passed
@kvaps
Andrei Kvapil (kvaps) deleted the revert/ingress-loadbalancer-mode branch May 19, 2026 08:47
Aleksei Sviridkin (lexfrei) added a commit that referenced this pull request Jun 30, 2026
…s + ServiceExposure) (#3081)

## What this PR does

Lays the foundation for migrating tenant LoadBalancer exposure off the
deprecated `Service.spec.externalIPs` (KEP-5707) and, eventually, off
MetalLB onto Cilium-native announcement — the mechanism-agnostic shape
prescribed when the earlier chart-side approach (#2468) was reverted in
#2603.

It adds a new `network.cozystack.io/v1alpha1` API group and a controller
that translates intent into whichever LoadBalancer backend the cluster
admin configured:

- `ExposureClass` (cluster-scoped, admin): binds a logical class name to
a backend (`externalIPs` | `metallb` | `cilium` | `robotlb`) and its VIP
address scope — the StorageClass/IngressClass pattern.
- `ServiceExposure` (namespaced, tenant intent): "expose this Service
via class X"; status reports the resolved backend and assigned IPs.
- A `serviceexposure` controller renders the backend's pool + announcer
once per class (MetalLB `IPAddressPool` + `L2Advertisement`; Cilium
`CiliumLoadBalancerIPPool` + `CiliumL2AnnouncementPolicy`), scoped to
the namespaces that opted in. Backends are pluggable behind a small Go
interface, so adding BGP or another cloud LB is one implementation, not
an API change. The controller never mutates the target Service — the
chart still owns its shape.

Platform wiring: `publishing.exposureClass.*` renders the admin
`ExposureClass` and the host ingress publishes through it. The default
(no class set, `externalIPs`) leaves every existing cluster's behaviour
byte-for-byte unchanged; the MetalLB→Cilium announcer migration becomes
a single `ExposureClass.spec.backend` change.

This is the foundation only — Part of #2586. Converting per-app
exposure, flipping Cilium `l2announcements` on, changing the default,
and removing MetalLB are deferred to later phases.

### Release note

```release-note
feat(network): add ExposureClass and ServiceExposure for mechanism-agnostic LoadBalancer exposure. Opt-in foundation; the default externalIPs publishing path is unchanged.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added new `ExposureClass` and `ServiceExposure` custom resources with
support for multiple backends (externalIPs, MetalLB, Cilium, RobotLB),
including optional L2 and address configuration.
* Added Helm wiring and templates to enable service/ingress publishing
via `ServiceExposure`, selecting a configured (or default)
`ExposureClass`.
* **Bug Fixes**
* Improved reconciliation, finalizer-based cleanup, and status updates
for readiness and assigned IP reporting, including safer handling when
backends or managed resources change.
* **Tests**
* Added end-to-end and unit test coverage for backend rendering,
readiness, and cleanup flows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/networking Issues or PRs related to networking (ingress, gateway, vpn, metallb, kube-ovn) kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/breaking-change Indicates the change introduces a breaking API or behaviour change size/L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant