Skip to content

fix(build): read --cache-from from ghcr :latest so PR builds warm-start - #2855

Merged
Aleksei Sviridkin (lexfrei) merged 1 commit into
mainfrom
fix/ci-cache-from-ghcr
Jun 9, 2026
Merged

fix(build): read --cache-from from ghcr :latest so PR builds warm-start#2855
Aleksei Sviridkin (lexfrei) merged 1 commit into
mainfrom
fix/ci-cache-from-ghcr

Conversation

@myasnikovdaniil

@myasnikovdaniil myasnikovdaniil commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

What this PR does

#2711 moved PR builds to unique pr-<N>-<sha> tags (PUBLISH_FLOATING=0) to end concurrent-push 409 conflicts, but left --cache-from pointing at $(REGISTRY)/<img>:latest. PR builds set REGISTRY to the per-CI registry, which never publishes :latest, so every PR build's registry cache lookup 404s and rebuilds cold — slowing builds and, on a loaded runner, pushing them into the 30-minute job timeout.

This adds a CACHE_REGISTRY knob (default ghcr.io/cozystack/cozystack, where release builds do publish :latest) and points every --cache-from at it. Push targets ($(REGISTRY)) and the unique per-PR tags are unchanged, so #2711's anti-conflict behaviour is preserved while PR builds regain a warm cache from the last release.

  • hack/common-envs.mk: new CACHE_REGISTRY ?= ghcr.io/cozystack/cozystack with rationale.
  • 31 package Makefiles: 37 --cache-from type=registry,ref=$(REGISTRY)/…$(CACHE_REGISTRY)/….

Verified with make -n image REGISTRY=iad.ocir.io/…: push tag stays on OCIR (…/cozystack-controller:pr-<N>-<sha>), cache-from now reads ghcr.io/cozystack/cozystack/cozystack-controller:latest. Makefile-only change — no generated artifacts affected.

Note: cache freshness is bounded by the last release's :latest. PR-to-PR freshness (a main-built :buildcache tag) is a possible follow-up, intentionally out of scope here.

Release note

fix(build): PR CI image builds now read layer cache from ghcr.io `:latest` (published by releases) instead of the per-CI registry where `:latest` was never pushed after #2711, restoring warm-start caching for PR builds.

Summary by CodeRabbit

  • Chores
    • Updated Docker build configuration across multiple components to use a dedicated cache registry for improved build layer reuse.

PR #2711 switched PR builds to push only :pr-<N>-<sha> tags (PUBLISH_FLOATING=0) to stop concurrent-push 409 conflicts, but left --cache-from pointing at $(REGISTRY)/<img>:latest. PR builds set REGISTRY to the per-CI registry (iad.ocir.io) and never publish :latest there, so every PR build's registry cache lookup 404s and rebuilds cold.

Add a CACHE_REGISTRY knob (default ghcr.io/cozystack/cozystack, where release builds do publish :latest) and point every --cache-from at it. Push targets ($(REGISTRY)) and the unique per-PR tags are unchanged, so the #2711 anti-conflict behaviour is preserved while PR builds regain a warm cache from the last release.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
@github-actions github-actions Bot added size/M This PR changes 30-99 lines, ignoring generated files area/build Issues or PRs related to image build infrastructure, multi-arch support labels Jun 9, 2026
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request improves CI build performance by decoupling the registry used for pushing PR images from the registry used for pulling build caches. Previously, PR builds were attempting to pull caches from a per-CI registry that did not contain the necessary :latest tags, resulting in cold builds and increased job duration. By introducing a CACHE_REGISTRY pointing to a central location where :latest tags are consistently published, PR builds can now effectively utilize cached layers.

Highlights

  • Build Cache Optimization: Introduced a new CACHE_REGISTRY variable to allow PR builds to fetch layer caches from a stable registry instead of the per-CI registry.
  • Makefile Updates: Updated 31 package Makefiles to point --cache-from references to the new CACHE_REGISTRY variable, ensuring PR builds can benefit from warm caches published by release builds.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment Gemini (@gemini-code-assist) Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@github-actions github-actions Bot added the kind/bug Categorizes issue or PR as related to a bug label Jun 9, 2026
@coderabbitai

coderabbitai Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 37dbd6d5-9aea-4902-942c-8c48d480074b

📥 Commits

Reviewing files that changed from the base of the PR and between 80fbfa1 and 9868c43.

📒 Files selected for processing (32)
  • hack/common-envs.mk
  • packages/apps/clickhouse/Makefile
  • packages/apps/http-cache/Makefile
  • packages/apps/kubernetes/Makefile
  • packages/apps/mariadb/Makefile
  • packages/core/installer/Makefile
  • packages/core/platform/Makefile
  • packages/core/talos/Makefile
  • packages/core/testing/Makefile
  • packages/extra/monitoring/Makefile
  • packages/system/backup-controller/Makefile
  • packages/system/backupstrategy-controller/Makefile
  • packages/system/bucket/Makefile
  • packages/system/cilium/Makefile
  • packages/system/cozystack-api/Makefile
  • packages/system/cozystack-controller/Makefile
  • packages/system/dashboard/Makefile
  • packages/system/flux-plunger/Makefile
  • packages/system/grafana-operator/Makefile
  • packages/system/kamaji/Makefile
  • packages/system/keycloak-operator/Makefile
  • packages/system/kilo/Makefile
  • packages/system/kubeovn-plunger/Makefile
  • packages/system/kubeovn-webhook/Makefile
  • packages/system/lineage-controller-webhook/Makefile
  • packages/system/linstor-gui/Makefile
  • packages/system/linstor/Makefile
  • packages/system/metallb/Makefile
  • packages/system/monitoring/Makefile
  • packages/system/multus/Makefile
  • packages/system/objectstorage-controller/Makefile
  • packages/system/redis-operator/Makefile

📝 Walkthrough

Walkthrough

This PR introduces a CACHE_REGISTRY variable in the build configuration and updates all Docker buildx invocations across the codebase to use it for --cache-from, separating the build cache source from the push registry destination.

Changes

Build Cache Registry Configuration

Layer / File(s) Summary
CACHE_REGISTRY variable definition
hack/common-envs.mk
Introduces CACHE_REGISTRY variable defaulting to ghcr.io/cozystack/cozystack with documentation explaining that cache sources should point to a registry where :latest is maintained, since PR builds publish only unique pr-<N>-<sha> tags without updating :latest.
Update image build cache sources
packages/apps/clickhouse/Makefile, packages/apps/http-cache/Makefile, packages/apps/kubernetes/Makefile, packages/apps/mariadb/Makefile, packages/core/installer/Makefile, packages/core/platform/Makefile, packages/core/talos/Makefile, packages/core/testing/Makefile, packages/extra/monitoring/Makefile, packages/system/backup-controller/Makefile, packages/system/backupstrategy-controller/Makefile, packages/system/bucket/Makefile, packages/system/cilium/Makefile, packages/system/cozystack-api/Makefile, packages/system/cozystack-controller/Makefile, packages/system/dashboard/Makefile, packages/system/flux-plunger/Makefile, packages/system/grafana-operator/Makefile, packages/system/kamaji/Makefile, packages/system/keycloak-operator/Makefile, packages/system/kilo/Makefile, packages/system/kubeovn-plunger/Makefile, packages/system/kubeovn-webhook/Makefile, packages/system/lineage-controller-webhook/Makefile, packages/system/linstor-gui/Makefile, packages/system/linstor/Makefile, packages/system/metallb/Makefile, packages/system/monitoring/Makefile, packages/system/multus/Makefile, packages/system/objectstorage-controller/Makefile, packages/system/redis-operator/Makefile
All Docker buildx --cache-from arguments are updated to use $(CACHE_REGISTRY) instead of $(REGISTRY), allowing build cache to be sourced independently from the push registry destination.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • cozystack/cozystack#932: Modifies overlapping Makefile build invocations but changes different arguments (--builder and --platform for cross-arch builds).

Suggested labels

kind/cleanup, size/L, area/ci, area/build

Suggested reviewers

  • lllamnyp
  • androndo
  • NickVolynkin
  • lexfrei

Poem

🐰 A registry for caches, another for push,
No more the same source, we've whisked through the brush,
Thirty-five targets now separated with care,
Layer warmth flows from :latest everywhere!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title accurately captures the main objective: fixing CI builds to read cache from ghcr.io :latest for warm-start caching in PR builds. It directly addresses the core change across all Makefiles.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/ci-cache-from-ghcr

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a CACHE_REGISTRY variable in hack/common-envs.mk and updates multiple package Makefiles to use it in the --cache-from argument of docker buildx build. This decouples the build cache source registry from the target push registry, allowing pull request builds to leverage warm caches from the main registry and avoid 404 errors. There are no review comments to evaluate, and I have no feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — surgical, correct fix; premise verified against source, change is complete and consistent, and #2711's anti-conflict tagging is preserved.

Business context: PR builds set REGISTRY to a per-CI registry that never publishes :latest (PUBLISH_FLOATING=0), so the old --cache-from $(REGISTRY)/<img>:latest always 404'd and rebuilt cold — pushing loaded runners into the 30-min timeout. This points --cache-from at ghcr, where releases do publish :latest.

Verified:

  • Premise holds. pull-requests.yaml sets REGISTRY to a per-CI registry with PUBLISH_VERSIONED=0/PUBLISH_FLOATING=0; per image-tags in common-envs.mk, :latest is only pushed when PUBLISH_FLOATING=1, so it never lands on the per-CI registry. tags.yaml sets both to 1 with the default ghcr REGISTRY, so releases publish :latest/:<version> to ghcr — a valid warm cache source.
  • Completeness. All 37 --cache-from refs across 31 Makefiles now use $(CACHE_REGISTRY); zero $(REGISTRY) cache-from refs remain. cache-to (inline) and push targets ($(REGISTRY)) are untouched, so #2711's unique pr-<N>-<sha> tagging is preserved.
  • Correctness. Inline cache is digest-matched, so reading cache from a registry different from the push target is safe — it can only change build speed, never output.
  • The non-:latest case (ubuntu-container-disk:$(ver)) is consistent: releases publish that versioned tag to ghcr under PUBLISH_VERSIONED=1.

Non-blocking follow-ups:

  1. Downstream/fork builders who set a custom REGISTRY and relied on cache-from reading their own registry now default to upstream ghcr; they must also set CACHE_REGISTRY. It's a ?= override, so this is minor — worth a line in build docs if any exist.
  2. The deferred :buildcache-from-main tag (already noted as out of scope) would restore PR-to-PR cache freshness between releases.

@lexfrei
Aleksei Sviridkin (lexfrei) merged commit 5d72a32 into main Jun 9, 2026
8 of 9 checks passed
@lexfrei
Aleksei Sviridkin (lexfrei) deleted the fix/ci-cache-from-ghcr branch June 9, 2026 09:37
IvanHunters added a commit to kvaps/cozystack that referenced this pull request Jun 9, 2026
Follows cozystack#2855. The talos-csr-signer image target was added on this
branch before cozystack#2855 merged, so the rebase brought it in still
pointing --cache-from at $(REGISTRY)/talos-csr-signer:latest, which
on PR CI is the per-CI registry where :latest is never published.
That cache lookup 404s on every PR build and forces a cold rebuild,
exactly the failure mode cozystack#2855 fixed for the other Makefiles.

Aligns the only remaining $(REGISTRY)-targeted --cache-from with the
$(CACHE_REGISTRY) ghcr.io convention from cozystack#2855 so PR builds of this
target warm-start off the last release's :latest.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/build Issues or PRs related to image build infrastructure, multi-arch support kind/bug Categorizes issue or PR as related to a bug size/M This PR changes 30-99 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants