Skip to content

[tenant] fix: allow egress to vlogs in parent tenants - #1971

Closed
Aleksei Sviridkin (lexfrei) wants to merge 1 commit into
mainfrom
fix/tenant-egress-vlogs
Closed

[tenant] fix: allow egress to vlogs in parent tenants#1971
Aleksei Sviridkin (lexfrei) wants to merge 1 commit into
mainfrom
fix/tenant-egress-vlogs

Conversation

@lexfrei

@lexfrei Aleksei Sviridkin (lexfrei) commented Feb 3, 2026

Copy link
Copy Markdown
Contributor

What this PR does

Adds vlogs to the tenant egress CiliumClusterwideNetworkPolicy, mirroring the existing vminsert rule.

Currently, nested Kubernetes clusters can send metrics to parent cluster's vminsert (explicitly allowed), but logs to vlogs are blocked by Cilium because there's no matching egress rule.

This causes Fluent Bit connection timeouts:

[error] [upstream] connection to tcp://10.43.81.27:9428 timed out
[error] [output:http:http.0] no upstream connections available to vlogs-generic.tenant-root.svc:9428

Fixes #1970

Release note

[tenant] Fixed log collection from nested Kubernetes clusters by allowing egress traffic to VLogs in parent tenants.

Summary by CodeRabbit

  • Chores
    • Updated network policy to support dynamic namespace matching for multi-tenant deployments.
    • Switched to expression-based label matching to allow multiple namespace prefixes.
    • Extended app matching to include both "vminsert" and "vlogs" workloads.
    • Adjusted conditional markers to preserve existing policy structure while enabling the new logic.

@dosubot dosubot Bot added the size/S This PR changes 10-29 lines, ignoring generated files label Feb 3, 2026
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello Aleksei Sviridkin (@lexfrei), I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical connectivity issue preventing nested Kubernetes clusters from sending logs to parent tenant vlogs services. By extending the existing network policy to explicitly permit egress traffic to vlogs, similar to how vminsert is handled, it ensures proper log collection and resolves connection timeouts experienced by Fluent Bit. This change is crucial for maintaining observability across tenant boundaries.

Highlights

  • Network Policy Update for VLogs: Modified the CiliumClusterwideNetworkPolicy to include egress rules specifically for vlogs services, mirroring the existing vminsert rule.
  • Nested Tenant Log Collection Fix: Enabled log collection from nested Kubernetes clusters by allowing egress traffic to vlogs in parent tenants, resolving Fluent Bit connection timeouts caused by blocked connections.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • packages/apps/tenant/templates/networkpolicy.yaml
    • Added a new toEndpoints rule within the CiliumClusterwideNetworkPolicy to allow egress traffic to vlogs services for non-root tenants.
    • The new rule uses matchLabels to target pods with k8s:app.kubernetes.io/name: "vlogs" and dynamically determines the target namespace based on the release namespace, ensuring correct routing to parent tenant vlogs.
Activity
  • The pull request was created by lexfrei to fix an issue with log collection from nested Kubernetes clusters.
  • The author provided a detailed description of the problem, including specific error messages (Fluent Bit connection timeouts), and outlined the proposed solution.
  • A release note was included, summarizing the fix for log collection from nested Kubernetes clusters.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment Gemini (@gemini-code-assist) Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@dosubot dosubot Bot added the kind/bug Categorizes issue or PR as related to a bug label Feb 3, 2026
@coderabbitai

coderabbitai Bot commented Feb 3, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

A conditional block was added to the tenant network policy template to allow egress traffic to vlogs (in addition to vminsert) for non-tenant-root releases by iterating over namespace segments and generating corresponding match entries.

Changes

Cohort / File(s) Summary
Network Policy Template
packages/apps/tenant/templates/networkpolicy.yaml
Added a dynamic matchExpressions block (replacing a static matchLabels for vminsert) that includes vlogs or vminsert as allowed apps and matches pod namespaces via In for derived namespace prefixes. Adjusted template markers accordingly.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

🐰 I nibble at templates, swift and light,
I stitch in vlogs so logs take flight,
Nested clusters whisper, now they send,
From burrowed pods to vlogs — logs mend. 🥕✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: allowing egress to vlogs in parent tenants, which matches the primary objective of the PR.
Linked Issues check ✅ Passed The PR successfully addresses issue #1970 by modifying the network policy to allow egress to vlogs, mirroring the existing vminsert rule.
Out of Scope Changes check ✅ Passed All changes are scoped to fixing the tenant egress policy for vlogs; no unrelated modifications are present.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch fix/tenant-egress-vlogs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request correctly adds an egress rule to the CiliumClusterwideNetworkPolicy to allow traffic to vlogs in parent tenants, fixing an issue with log collection from nested clusters. The change mirrors the existing rule for vminsert. However, this introduces code duplication. My review includes a suggestion to refactor the policy to combine the rules for vminsert and vlogs, which would improve the maintainability of this Helm template.

Comment on lines +56 to +68
{{- if ne (include "tenant.name" .) "tenant-root" }}
- toEndpoints:
{{- if hasPrefix "tenant-" .Release.Namespace }}
{{- $parts := splitList "-" .Release.Namespace }}
{{- range $i, $v := $parts }}
{{- if ne $i 0 }}
- matchLabels:
"k8s:app.kubernetes.io/name": "vlogs"
"k8s:io.kubernetes.pod.namespace": {{ join "-" (slice $parts 0 (add $i 1)) }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This block is a near-duplicate of the vminsert rule on lines 43-55, which harms maintainability. Instead of adding a new block, you can combine the logic for both vlogs and vminsert into a single rule.

I recommend deleting this new block and modifying the vminsert block to use matchExpressions, which can match multiple values for a label. This makes the policy more concise and easier to extend.

Here's how you could refactor the vminsert block to include vlogs:

  - toEndpoints:
    {{- if hasPrefix "tenant-" .Release.Namespace }}
    {{- $parts := splitList "-" .Release.Namespace }}
    {{- range $i, $v := $parts }}
    {{- if ne $i 0 }}
    - matchExpressions:
      - key: "k8s:io.kubernetes.pod.namespace"
        operator: "In"
        values:
        - {{ join "-" (slice $parts 0 (add $i 1)) | quote }}
      - key: "k8s:app.kubernetes.io/name"
        operator: "In"
        values:
        - "vminsert"
        - "vlogs"
    {{- end }}
    {{- end }}
    {{- end }}

This single toEndpoints rule would replace the two separate rules for vminsert and vlogs.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, refactored to use matchExpressions combining vminsert and vlogs into a single rule.

Combine vminsert and vlogs egress rules using matchExpressions
to reduce code duplication. This allows Fluent Bit in nested
Kubernetes clusters to send logs to VLogs in parent tenants.

Without this fix, logs from nested clusters timeout because Cilium
blocks traffic to vlogs pods while metrics to vminsert work fine.

Fixes #1970

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
@kvaps Andrei Kvapil (kvaps) added backport Should change be backported on previous release and removed backport Should change be backported on previous release labels Feb 4, 2026
@kvaps

Copy link
Copy Markdown
Member

This is probably a bug; we need to redesign this somehow (for example, by using a user-space proxy).
As it is now, this would allow tenant users to read logs from parent tenants. The same issue applies to metrics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/bug Categorizes issue or PR as related to a bug size/S This PR changes 10-29 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nested cluster Fluent Bit cannot send logs to parent VLogs: missing vlogs in tenant egress policy

2 participants