Skip to content

fix(platform): route OIDC backend calls through the in-cluster Keycloak by default - #3289

Merged
myasnikovdaniil merged 1 commit into
mainfrom
fix/oidc-keycloak-internal-url-default
Jul 15, 2026
Merged

fix(platform): route OIDC backend calls through the in-cluster Keycloak by default#3289
myasnikovdaniil merged 1 commit into
mainfrom
fix/oidc-keycloak-internal-url-default

Conversation

@lexfrei

@lexfrei Aleksei Sviridkin (lexfrei) commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

What this PR does

Defaults authentication.oidc.keycloakInternalUrl to http://keycloak-http.cozy-keycloak.svc:8080/realms/cozy instead of an empty string.

With the empty default, oauth2-proxy performed OIDC discovery against the external Keycloak hostname. That request resolves through public DNS and lands on the root ingress controller, which does not serve yet on a fresh install: the root tenant's IngressClass only exists once tenant-root is ready, and tenant-root is still reconciling while the dashboard starts. Discovery has nothing to talk to, the gatekeeper fails, and the dashboard never comes up — with no obvious diagnosis for a first-time installer.

The bundled Keycloak is the only supported provider (the external hostname is templated as keycloak.<host> and the realm is fixed), so backend-to-backend calls have no reason to leave the cluster. The new default is the same Service address keycloak-configure already uses for its own backend calls. The dashboard and linstor-gui gatekeepers then run with --skip-oidc-discovery and send token, jwks, userinfo and logout requests straight to the Service, while browser-facing redirects stay on the external host.

Both gatekeeper templates already branch on the value being non-empty, so neither needed a change. The value stays overridable for a Keycloak that does not live at the standard in-cluster address, and setting it back to "" restores the previous external-discovery behaviour.

Token validation is unaffected: Keycloak pins KC_HOSTNAME to the external URL, so tokens redeemed over the internal Service still carry the external iss and match the unchanged --oidc-issuer-url.

Upgrade behaviour

Existing OIDC clusters that never set keycloakInternalUrl pick up the new default on upgrade: their gatekeepers roll once and move to the internal path. Clusters that set the value explicitly — including to "" — keep exactly what they configured.

Security posture

The backend leg (token redemption, which carries the gatekeeper's Keycloak client secret, plus jwks/userinfo/logout) now travels as plaintext HTTP between pods rather than HTTPS to the ingress. This is stated in the values.yaml comment rather than left implicit. The keycloak-http Service has no TLS listener, so in-cluster HTTPS is not an available alternative, and keycloak-configure already sends Keycloak admin credentials over that identical address — anyone positioned to read this traffic can already reach a strictly more powerful secret. The externally reachable front door (browser to oauth2-proxy) is unchanged.

Testing

helm-unittest coverage across all three affected charts: the default landing on the _cluster channel, both gatekeepers rendering the in-cluster backend URLs while keeping browser URLs external, an explicit override winning, the empty-string escape hatch still rendering the external-discovery shape, and OIDC-disabled clusters (now the common case of "OIDC off, internal URL set") ignoring the value entirely.

The E2E OIDC case previously applied this exact URL as a manual workaround. It now sets only oidc.enabled and asserts the consequence: it waits for the auth-proxy container to appear in the gatekeeper Deployment, then gates on kubectl rollout status. Readiness alone would prove nothing there — the HelmRelease is still Ready from the token-proxy install, and with replicas: 1 and maxUnavailable: 25% (which rounds down to 0) the Deployment keeps the old ReplicaSet Available while a new pod crashloops. Only the rollout catches a regression back to external discovery.

Follow-up: the OIDC documentation in the website repo still frames keycloakInternalUrl as an optional tweak "for self-signed certificates or restricted external access". It is now the default path and needs a separate docs PR.

Closes #3288

Screenshots

No UI changes.

Release note

fix(platform): `authentication.oidc.keycloakInternalUrl` now defaults to the in-cluster Keycloak Service (`http://keycloak-http.cozy-keycloak.svc:8080/realms/cozy`). OIDC backend calls from the dashboard and linstor-gui gatekeepers no longer leave the cluster, so enabling OIDC on a fresh install no longer leaves the dashboard down while the root ingress is not yet serving. Existing clusters that never set the value pick up the new default and roll their gatekeepers once. Set the value explicitly to point at a non-standard Keycloak placement, or to an empty string to restore external OIDC discovery.

Summary by CodeRabbit

  • New Features

    • OIDC-enabled applications now use the internal Keycloak service for backend authentication by default, improving reliability and avoiding unnecessary external discovery.
    • Browser redirects and login flows continue using the external Keycloak address.
    • Operators can provide a custom internal Keycloak URL or set it to an empty value to restore external OIDC discovery.
  • Bug Fixes

    • Dashboard and Linstor GUI authentication now correctly apply internal Keycloak settings only when OIDC is enabled.
    • Dashboard upgrades now wait for authentication components to become ready before completing.

@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: d17cdc64-da7a-49ae-8c08-8e79c6f860bf

📥 Commits

Reviewing files that changed from the base of the PR and between 6f58898 and f24b563.

📒 Files selected for processing (5)
  • hack/e2e-install-cozystack.bats
  • packages/core/platform/tests/apps_oidc_keycloak_internal_url_test.yaml
  • packages/core/platform/values.yaml
  • packages/system/dashboard/tests/gatekeeper_test.yaml
  • packages/system/linstor-gui/tests/ingress_auth_test.yaml

📝 Walkthrough

Walkthrough

The platform now defaults OIDC backend requests to the in-cluster Keycloak URL. Dashboard and Linstor GUI gatekeepers support default, explicit, and empty-string URL modes, with expanded rendering tests and stronger end-to-end rollout validation.

Changes

OIDC internal Keycloak routing

Layer / File(s) Summary
Platform OIDC URL contract
packages/core/platform/values.yaml, packages/core/platform/tests/apps_oidc_keycloak_internal_url_test.yaml
The default internal Keycloak realm URL is documented and tested alongside explicit overrides and the empty-string fallback.
Dashboard gatekeeper routing
packages/system/dashboard/tests/gatekeeper_test.yaml
Tests validate OIDC-disabled behavior and auth-proxy arguments for internal routing, custom overrides, and external discovery.
Linstor GUI and install validation
packages/system/linstor-gui/tests/ingress_auth_test.yaml, hack/e2e-install-cozystack.bats
Linstor GUI routing modes are covered, and the install test waits for the OIDC gatekeeper rollout and dashboard HelmRelease readiness.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant AuthProxy
  participant KeycloakService
  participant ExternalIssuer
  AuthProxy->>KeycloakService: Backend token and discovery-related requests
  AuthProxy->>ExternalIssuer: Browser-facing redirects and issuer URLs
  AuthProxy->>KeycloakService: Use internal URL with discovery skipped
Loading

Possibly related PRs

Suggested labels: area/testing

Suggested reviewers: kvaps, lllamnyp, androndo, ivanhunters, sircthulhu

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: routing OIDC backend calls through in-cluster Keycloak by default.
Linked Issues check ✅ Passed The PR matches #3288 by defaulting Keycloak backend calls in-cluster, keeping overrides and empty-string escape hatch, preserving browser redirects, and covering both gatekeepers.
Out of Scope Changes check ✅ Passed The modified files all support the OIDC Keycloak default/override behavior and related tests, with no obvious unrelated scope.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/oidc-keycloak-internal-url-default

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added area/platform Issues or PRs related to platform infrastructure (bundle, flux, talos, installer) kind/bug Categorizes issue or PR as related to a bug size/L This PR changes 100-499 lines, ignoring generated files labels Jul 14, 2026
…ak by default

authentication.oidc.keycloakInternalUrl defaulted to an empty string, which
left oauth2-proxy performing OIDC discovery against the external Keycloak
hostname. That request resolves through public DNS and lands on the root
ingress controller, which does not serve yet on a fresh install: the root
tenant's IngressClass only exists once tenant-root is ready, and tenant-root
is still reconciling while the dashboard starts. Discovery has nothing to talk
to, the gatekeeper fails, and the dashboard never comes up.

The bundled Keycloak is the only supported provider — the external hostname is
templated as keycloak.<host> and the realm is fixed — so backend-to-backend
calls have no reason to leave the cluster. Default the value to
http://keycloak-http.cozy-keycloak.svc:8080/realms/cozy, the same Service
address keycloak-configure already uses. The dashboard and linstor-gui
gatekeepers then run with --skip-oidc-discovery and send token, jwks, userinfo
and logout requests to the Service, while browser redirects stay on the
external host. Those calls travel as plaintext HTTP between pods, as
keycloak-configure's admin calls to the same address already do.

The value remains overridable for a Keycloak that does not live at the standard
in-cluster address, and setting it back to an empty string restores external
discovery. Existing OIDC clusters that never set it get a gatekeeper rollout
onto the internal path on upgrade.

Enabling OIDC swaps the dashboard's token-proxy container for oauth2-proxy, so
the E2E install now drives that swap to completion: it waits for the auth-proxy
container to appear, then gates on the gatekeeper rollout. Readiness alone
proves nothing there — the HelmRelease is still Ready from the token-proxy
install, and the Deployment keeps the old ReplicaSet Available while a new pod
crashloops, so only the rollout catches a regression back to external discovery.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request optimizes OIDC authentication by routing backend-to-backend calls through the internal Keycloak Service by default. This change improves system stability during initial cluster deployment by decoupling the dashboard's readiness from the external ingress controller. The configuration remains fully backward compatible and overridable for custom setups.

Highlights

  • Default Configuration Change: Updated the default value for authentication.oidc.keycloakInternalUrl to point to the in-cluster Keycloak Service, ensuring backend OIDC calls remain within the cluster.
  • Improved Reliability: Prevents dashboard startup failures on fresh installs by removing the dependency on the external Keycloak ingress being ready for OIDC discovery.
  • Testing and Validation: Added comprehensive helm-unittest coverage and updated E2E tests to verify the new default behavior, override capabilities, and the empty-string escape hatch.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment Gemini (@gemini-code-assist) Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the default value of keycloakInternalUrl in the platform package to point to the in-cluster Keycloak Service address. This change allows backend-to-backend OIDC requests to bypass external DNS and root ingress, preventing dashboard startup failures on fresh installations. The PR also adds extensive unit tests for the platform, dashboard, and linstor-gui packages to verify this behavior, and updates the E2E installation tests to validate the rollout of the gatekeeper with the new default. The feedback suggests regenerating the values.schema.json file by running make generate to keep it in sync with the updated default value in values.yaml.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

#
# Override only for a KeyCloak that does not live at the standard in-cluster address. Setting it
# to an empty string restores full OIDC discovery against the external issuer.
keycloakInternalUrl: "http://keycloak-http.cozy-keycloak.svc:8080/realms/cozy"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since the default value of keycloakInternalUrl has been updated in values.yaml, the corresponding values.schema.json file needs to be regenerated to stay in sync. Please run make generate to update the schema file.

@myasnikovdaniil myasnikovdaniil left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Minimal, reversible fix — it flips exactly one default (keycloakInternalUrl → the in-cluster Keycloak Service) and rides template branches that already existed. Verified against the tree: value wiring (apps.yaml:92_cluster → both gatekeepers read index ... "keycloak-internal-url" | default "" and gate the in-cluster args in one {{- if }} block, "" escape hatch intact); the issuer stays external (KC_HOSTNAME pins the external host so the token iss still matches the unchanged --oidc-issuer-url); and the plaintext backend leg is already precedented by keycloak-configure hitting the same keycloak-http...svc:8080. helm-unittest plus the reworked e2e (which gates on rollout status, not a vacuous condition=ready) cover it. One nit inline; the website OIDC docs are the tracked follow-up.

#
# Override only for a KeyCloak that does not live at the standard in-cluster address. Setting it
# to an empty string restores full OIDC discovery against the external issuer.
keycloakInternalUrl: "http://keycloak-http.cozy-keycloak.svc:8080/realms/cozy"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: a one-line upgrade note here would make values.yaml self-documenting — flipping this default silently rolls existing OIDC gatekeepers once on upgrade; set it to "" to keep the previous external-discovery behaviour. (Already captured in the PR body / release note.)

@myasnikovdaniil
myasnikovdaniil merged commit 6c9edf5 into main Jul 15, 2026
19 checks passed
@myasnikovdaniil
myasnikovdaniil deleted the fix/oidc-keycloak-internal-url-default branch July 15, 2026 08:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/platform Issues or PRs related to platform infrastructure (bundle, flux, talos, installer) kind/bug Categorizes issue or PR as related to a bug size/L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OIDC: dashboard cannot bootstrap because oauth2-proxy discovers Keycloak through an ingress that does not serve yet

2 participants