Skip to content

fix(kafka): set entity-operator resources to avoid OOM under tenant LimitRange - #2934

Merged
myasnikovdaniil merged 1 commit into
mainfrom
fix/kafka-entity-operator-resources
Jun 25, 2026
Merged

fix(kafka): set entity-operator resources to avoid OOM under tenant LimitRange#2934
myasnikovdaniil merged 1 commit into
mainfrom
fix/kafka-entity-operator-resources

Conversation

@myasnikovdaniil

@myasnikovdaniil myasnikovdaniil commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

What this PR does

packages/apps/kafka rendered the Strimzi Kafka CR with entityOperator.topicOperator: {} / userOperator: {} and no resources. Tenant namespaces ship a LimitRange that defaults containers to memory: 128Mi, so the topic-operator and user-operator JVM containers (Kafka 3.9.1 + fabric8/vertx) are OOMKilled (exit 137) on startup. The <cluster>-entity-operator Deployment never becomes Ready (CrashLoopBackOff), so:

  • KafkaTopic / KafkaUser CRs never reconcile — topic creation effectively fails;
  • the cluster-operator logs Exceeded timeout of 300000ms while waiting for Deployment ...-entity-operator ... to be ready;
  • on teardown, KafkaTopics get stuck permanently with the strimzi.io/topic-operator finalizer (no operator left to clear them), blocking namespace deletion.

The Kafka broker + ZooKeeper come up fine — only the entity-operator dies. The apps.cozystack.io/Kafka schema exposes no field for entity-operator resources, so there is no user-side workaround.

Fix

Set explicit resources (requests 256Mi/100m, limit 512Mi memory) on entityOperator.topicOperator and userOperator so they no longer inherit the 128Mi namespace default. Adds a helm-unittest assertion for the rendered limits.

Testing

Reproduced on a clean cluster (Cozystack v1.4.3 and v1.5.0-rc.2; Strimzi 0.45.1, Kafka 3.9.1): both entity-operator containers OOMKilled at 128Mi → CrashLoopBackOff → KafkaTopic never READY. After bumping to 512Mi the entity-operator rolls cleanly (both containers Ready, 0 restarts) and the KafkaTopic reconciles to READY=True.

Release note

fix(kafka): give entity-operator (topic/user-operator) explicit resources so they no longer inherit the 128Mi tenant LimitRange default and OOMKill, which left KafkaTopics/KafkaUsers unreconciled.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes

    • Kafka topic and user operators now use explicit CPU and memory requests and limits, avoiding reliance on namespace-level defaults.
  • Tests

    • Added regression coverage to verify the Kafka entity operator’s topic and user operator memory limits are explicitly set (512Mi).

@coderabbitai

coderabbitai Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: ebe7d76e-fb56-4970-bca7-7684c1a70946

📥 Commits

Reviewing files that changed from the base of the PR and between 3ee03da and 1ab5f17.

📒 Files selected for processing (2)
  • packages/apps/kafka/templates/kafka.yaml
  • packages/apps/kafka/tests/entityoperator_template_test.yaml
✅ Files skipped from review due to trivial changes (2)
  • packages/apps/kafka/tests/entityoperator_template_test.yaml
  • packages/apps/kafka/templates/kafka.yaml

📝 Walkthrough

Walkthrough

The Strimzi Kafka Helm template adds explicit CPU/memory requests and a 512Mi memory limit to both topicOperator and userOperator under entityOperator, replacing empty object placeholders. A corresponding test file adds assertions verifying these memory limits and the correct placement of enableServiceLinks.

Changes

Kafka entityOperator Resource Constraints

Layer / File(s) Summary
topicOperator/userOperator resource settings and test coverage
packages/apps/kafka/templates/kafka.yaml, packages/apps/kafka/tests/entityoperator_template_test.yaml
topicOperator and userOperator are changed from empty objects to explicit resource blocks with CPU/memory requests and a 512Mi memory limit; tests assert those limits and verify enableServiceLinks is at spec.entityOperator.template.pod with no phantom spec key present.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~5 minutes

Suggested reviewers

  • kvaps
  • lllamnyp
  • lexfrei

Poem

🐇 A rabbit once hopped through the JVM land,
Where limits were missing — oh, wasn't that grand?
Now 512Mi is set, neat and precise,
No LimitRange defaults to pay the full price.
The topicOperator breathes, userOperator too —
Bounded and tested, all shiny and new! 🎉

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and concisely describes the main change: setting entity-operator resource limits to prevent OOM issues under tenant LimitRange constraints.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/kafka-entity-operator-resources

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added area/database Issues or PRs related to managed databases (postgres, mariadb, redis, etcd, kafka, clickhouse) kind/bug Categorizes issue or PR as related to a bug size/M This PR changes 30-99 lines, ignoring generated files labels Jun 16, 2026
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses an issue where the Kafka entity-operator containers would crash due to insufficient memory when running in tenant namespaces with default LimitRanges. By explicitly defining resource requirements, the entity-operator can now start reliably, ensuring that KafkaTopic and KafkaUser resources are properly reconciled.

Highlights

  • Resource Allocation: Added explicit CPU and memory resource requests and limits to the Kafka entity-operator (topic and user operators) to prevent OOMKilled errors in environments with strict LimitRanges.
  • Testing: Included a new helm-unittest assertion to verify that the entity-operator resource limits are correctly rendered in the Kafka CR.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment Gemini (@gemini-code-assist) Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request configures explicit resource requests and limits for the topicOperator and userOperator in the Kafka entity operator template to prevent OOMKill issues caused by default LimitRanges in tenant namespaces. It also adds a corresponding test case to verify the memory limits. The reviewer suggests expanding the test assertions to also verify the CPU and memory requests for both operators.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +29 to +34
- equal:
path: spec.entityOperator.topicOperator.resources.limits.memory
value: 512Mi
- equal:
path: spec.entityOperator.userOperator.resources.limits.memory
value: 512Mi

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

To ensure comprehensive test coverage and prevent regressions, consider also asserting the CPU and memory requests for both the topicOperator and userOperator.

      - equal:
          path: spec.entityOperator.topicOperator.resources.requests.cpu
          value: 100m
      - equal:
          path: spec.entityOperator.topicOperator.resources.requests.memory
          value: 256Mi
      - equal:
          path: spec.entityOperator.topicOperator.resources.limits.memory
          value: 512Mi
      - equal:
          path: spec.entityOperator.userOperator.resources.requests.cpu
          value: 100m
      - equal:
          path: spec.entityOperator.userOperator.resources.requests.memory
          value: 256Mi
      - equal:
          path: spec.entityOperator.userOperator.resources.limits.memory
          value: 512Mi

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
packages/apps/kafka/tests/entityoperator_template_test.yaml (1)

24-34: ⚡ Quick win

Expand assertions to cover requested resources too.

The test currently validates only limits.memory, so it can miss regressions in requests.cpu and requests.memory even though those are part of the explicit resource contract.

Suggested test extension
   - it: topic-operator and user-operator set explicit resources (avoid 128Mi LimitRange OOM)
     release:
       name: test-kafka
       namespace: tenant-test
     asserts:
+      - equal:
+          path: spec.entityOperator.topicOperator.resources.requests.cpu
+          value: 100m
+      - equal:
+          path: spec.entityOperator.topicOperator.resources.requests.memory
+          value: 256Mi
       - equal:
           path: spec.entityOperator.topicOperator.resources.limits.memory
           value: 512Mi
+      - equal:
+          path: spec.entityOperator.userOperator.resources.requests.cpu
+          value: 100m
+      - equal:
+          path: spec.entityOperator.userOperator.resources.requests.memory
+          value: 256Mi
       - equal:
           path: spec.entityOperator.userOperator.resources.limits.memory
           value: 512Mi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/apps/kafka/tests/entityoperator_template_test.yaml` around lines 24
- 34, The test case "topic-operator and user-operator set explicit resources
(avoid 128Mi LimitRange OOM)" currently only validates limits.memory for both
topicOperator and userOperator, but does not verify the requested resources
(requests.cpu and requests.memory). Add four additional equal assertions to the
asserts section to validate
spec.entityOperator.topicOperator.resources.requests.cpu,
spec.entityOperator.topicOperator.resources.requests.memory,
spec.entityOperator.userOperator.resources.requests.cpu, and
spec.entityOperator.userOperator.resources.requests.memory with their expected
values to ensure complete coverage of the explicit resource contract and prevent
regressions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@packages/apps/kafka/tests/entityoperator_template_test.yaml`:
- Around line 24-34: The test case "topic-operator and user-operator set
explicit resources (avoid 128Mi LimitRange OOM)" currently only validates
limits.memory for both topicOperator and userOperator, but does not verify the
requested resources (requests.cpu and requests.memory). Add four additional
equal assertions to the asserts section to validate
spec.entityOperator.topicOperator.resources.requests.cpu,
spec.entityOperator.topicOperator.resources.requests.memory,
spec.entityOperator.userOperator.resources.requests.cpu, and
spec.entityOperator.userOperator.resources.requests.memory with their expected
values to ensure complete coverage of the explicit resource contract and prevent
regressions.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 4d983303-5a66-4164-ad8a-85f58b84883e

📥 Commits

Reviewing files that changed from the base of the PR and between 39f3ac7 and 3ee03da.

📒 Files selected for processing (2)
  • packages/apps/kafka/templates/kafka.yaml
  • packages/apps/kafka/tests/entityoperator_template_test.yaml

@myasnikovdaniil myasnikovdaniil self-assigned this Jun 23, 2026
…imitRange

entityOperator.topicOperator/userOperator carried no resources, so in tenant
namespaces (the LimitRange defaults containers to 128Mi) the JVM topic-operator
and user-operator are OOMKilled on startup. The entity-operator stays in
CrashLoopBackOff, KafkaTopics/KafkaUsers never reconcile, and on teardown topics
get stuck on the strimzi.io/topic-operator finalizer. Set explicit requests/limits
(256Mi/512Mi) so they no longer inherit the namespace default; add a template test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
@myasnikovdaniil
myasnikovdaniil force-pushed the fix/kafka-entity-operator-resources branch from 3ee03da to 1ab5f17 Compare June 23, 2026 08:46

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (APPROVE)

Business context: entityOperator.topicOperator/userOperator rendered without resources, so under the tenant LimitRange default (memory: 128Mi) the topic-operator and user-operator JVM containers OOMKill and never reconcile KafkaTopic/KafkaUser. This PR sets explicit resources so they no longer inherit that default.

Blockers: none.

The problem is real and the fix is plumbed correctly:

  • The tenant LimitRange applies default.memory: 128Mi to every container, which the JVM operator containers inherit and OOM on. Evidence: packages/apps/tenant/templates/quota.yaml:18-26.
  • resources is a valid field under both topicOperator and userOperator in kafka.strimzi.io/v1beta2, so the values reach the operator containers. Evidence: packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/040-Crd-kafka.yaml:3463,3653.
  • The chosen limit cannot break scheduling: that LimitRange defines only default/defaultRequest — no max, min, or maxLimitRequestRatio — so 256Mi/512Mi has no upper bound to violate, and 512Mi sits well above the 128Mi that OOMs (and in line with the cluster-operator's own 384Mi). Evidence: packages/apps/tenant/templates/quota.yaml:16-26.
  • helm-unittest passes (35/35), including the new assertion.

Non-blocking:

  1. The test asserts only limits.memory for each operator. The fix is the whole resources block, so requests.cpu/requests.memory are part of the contract and currently unguarded — a later edit could drop them without failing the suite. Add equal assertions for resources.requests.cpu (100m) and resources.requests.memory (256Mi) on both operators. Evidence: packages/apps/kafka/tests/entityoperator_template_test.yaml:31-34.
  2. The resources are hardcoded with no values knob. That matches the platform's sensible-default approach and is fine here; exposing entity-operator sizing the way kafka/zookeeper expose resourcesPreset would be a reasonable follow-up if tuning is ever needed. Evidence: packages/apps/kafka/values.yaml (no entityOperator section).

@myasnikovdaniil myasnikovdaniil added backport Should change be backported on previous release backport-previous Backport target — previous release line labels Jun 25, 2026
@myasnikovdaniil
myasnikovdaniil merged commit 59ded0a into main Jun 25, 2026
13 checks passed
@myasnikovdaniil
myasnikovdaniil deleted the fix/kafka-entity-operator-resources branch June 25, 2026 05:30
@github-actions

Copy link
Copy Markdown

Backport failed for release-1.4, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin release-1.4
git worktree add -d .worktree/backport-2934-to-release-1.4 origin/release-1.4
cd .worktree/backport-2934-to-release-1.4
git switch --create backport-2934-to-release-1.4
git cherry-pick -x 1ab5f17caca0a950596f6dfd6e64eda318739e1e

@github-actions

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/database Issues or PRs related to managed databases (postgres, mariadb, redis, etcd, kafka, clickhouse) backport Should change be backported on previous release backport-previous Backport target — previous release line kind/bug Categorizes issue or PR as related to a bug size/M This PR changes 30-99 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants