NextBSD — drop CI caching & purge caches Plan ready

Remove every caching layer from the nextbsd-redux build chain and purge the stored caches — via direct commits, in build-chain order. Plus the correction that matters: this alone does not fix the “freebsd-version stuck at 15.0” symptom — that is a version-pin / stale-image issue, not a cache-replay one.

TL;DR

Three cache mechanisms span the chain: ccache (actions/cache of .ccache) in nextbsd-kernel, nextbsd-freebsd-compat, nextbsd-kernel-modules; a distfiles cache in nextbsd; and a Docker BuildKit GHA layer cache in nextbsd-kernel-toolchain. Dropping them is mechanical — except the ccache repos wire the cache into the build via --cross-bindir=${CCACHE_CROSS_BINDIR}, so each make.py call must be repointed to ${CROSS_BINDIR} (the real Clang 19), not just have its cache step deleted.

Premise correction: the 15.0 stickiness was not a ccache replay. The kernel sets RELEASE=$(date -u) (a timestamp) and leaves REVISION stock at 15.0 by design; vers.c changes every build, so ccache always misses on it. The real 15.0 came from a stale baked toolchain image plus genuine version pins (nextbsd-kernel-modules on releng/15.0, nextbsd/build.sh default 15.0). Purging Actions caches does not touch the GHCR toolchain image or those pins.

Do both, but know what each buys: dropping cache = determinism + removes a future stale-state foot-gun (worth doing). Fixing freebsd-version = rebuild/refresh the toolchain image + the 15.1 base bump (tracked with the drm-kmod 6.12 / FreeBSD 15.1 work).

1. Two truths to keep separate

Drop cache + purge requestedFix freebsd-version → 15.1 separate
What it isRemove ccache / distfiles / BuildKit GHA cache; delete stored cachesRefresh the GHCR toolchain image; bump the real 15.0 version pins
What it fixesNon-determinism; stale-state surprises on future bumps; build hygieneThe actual reported-version stickiness
Does the other one?No — cache purge cannot change a baked image tag or a releng/15.0 pinIndependent of caching

This plan covers the requested cache work in full (§2–§4) and documents the version fix (§5) so the two are not conflated again.

2. Cache inventory (nextbsd-redux)

RepoCacheMechanism
nextbsd-kernelccacheactions/cache@v4 of .ccache, key ccache-kernel-${target}-${run_id} + prefix restore-keys
nextbsd-freebsd-compatccachesame, ccache-base-${target}-… (8 GiB)
nextbsd-kernel-modulesccachesame, ccache-modules-${target}-…
nextbsddistfilesactions/cache@v4 of distfiles, key on hashFiles(pkglist.txt,…) (build.yml:117-121)
nextbsd-kernel-toolchainBuildKit layerDocker cache-from/cache-to: type=gha (build-toolchain.yml:95-96); also publishes the GHCR image downstream pulls

freebsd-src (publish-only) and nextbsd-redux.github.io cache nothing — out of scope. The local nextbsd-ci/ directory is a stale, non-git scratch copy (pinned releng/15.0); it is not a repo and not in scope.

3. Drop recipe (per repo)

3a. ccache repos — delete steps and repoint the compiler

Load-bearing: every tools/build/make.py call passes --cross-bindir="${CCACHE_CROSS_BINDIR}" (/opt/ccache-cross, a wrapper dir baked in the toolchain image). Deleting only the cache step leaves the flag pointing at the wrapper. Repoint each call to --cross-bindir="${CROSS_BINDIR}" (/usr/lib/llvm-19/bin, the real Clang 19 the wrapper mirrors — and exactly what the Dockerfile builds the toolchain with). Do not drop the flag: make.py would then build its own LLVM from source.

In each repo: remove the job-level env: CCACHE_DIR, the Restore ccache step, the ccache reset counters step, and the ccache stats step; then repoint every --cross-bindir:

RepoDeleteRepoint --cross-bindir (CCACHE_CROSS_BINDIR → CROSS_BINDIR)
nextbsd-kernelenv CCACHE_DIR (l.21); Restore (l.42-48); reset (l.50-54); stats (l.227-228)lines 112, 220
nextbsd-freebsd-compatenv (l.27); Restore (l.44-50); reset (l.52-56); stats (l.70-71)lines 63, 66, 182
nextbsd-kernel-modulesenv (l.34); Restore (l.66-72); reset (l.74-78); stats (l.106-107)lines 99, 230 (the drm-kmod job has no ccache steps — only the l.230 repoint)

3b. nextbsd — distfiles cache

Delete the single step at build.yml:117-121 (the only actions/cache in the file). distfiles then re-fetch from the pkg mirrors inside the VM each run — a modest, network-bound cost well under the 120-min timeout. Nothing else references the distfiles path.

3c. nextbsd-kernel-toolchain — BuildKit GHA layer cache

Remove cache-from: type=gha,… and cache-to: type=gha,mode=max,… at build-toolchain.yml:95-96 so every image build is from scratch off current releng/15.1 freebsd-src. (Optional follow-up: drop the now-unused /opt/ccache-cross build block + CCACHE_CROSS_BINDIR ENV from Dockerfile.{amd64,arm64} once the workflows stop referencing it.)

4. Purge the stored caches

gh v2.91 has built-in gh cache delete --all. Caches observed live (read-only):

# nextbsd:                 freebsd-15.1-2.2.1-amd64-v2, distfiles-15.1-RELEASE-amd64-<hash>
# nextbsd-freebsd-compat:   ccache-base-{amd64,arm64}-<run>
# nextbsd-kernel:           ccache-kernel-{amd64,arm64}-<run>
# nextbsd-kernel-modules:   ccache-modules-{amd64,arm64}-<run>
# nextbsd-kernel-toolchain: index-{amd64,arm64}-*, buildkit-blob-* (~2 GiB)

for r in nextbsd nextbsd-freebsd-compat nextbsd-kernel nextbsd-kernel-modules nextbsd-kernel-toolchain; do
  gh cache delete --all -R nextbsd-redux/$r --succeed-on-no-caches
done

REST fallback: iterate gh api /repos/nextbsd-redux/<repo>/actions/caches then gh api --method DELETE …/actions/caches?key=<key>.

GHCR image is not an Actions cache. The toolchain’s :<arch>-latest / :<arch>-fbsd-<sha> GHCR tags are what downstream actually pulls; purging Actions caches does not refresh them. To get a clean toolchain: delete the BuildKit GHA cache (above), then re-run build-toolchain.yml (workflow_dispatch), or delete stale GHCR versions: gh api --method DELETE /orgs/nextbsd-redux/packages/container/nextbsd-kernel-toolchain/versions/<id> (needs delete:packages scope).

5. The actual freebsd-version fix (cache purge alone won’t do it)

Root cause, verified in source:

By design, leave alone: kernel REVISION=15.0 is intentional (protects __FreeBSD__=15 and the compiler target triple). The 15.1 move for nextbsd-kernel-modules is gated on the drm-kmod 6.6→6.12 / FreeBSD 15.1 effort — bump releng/15.0→15.1 there as part of that, not as cache work.

6. Execution — direct commits, build-chain order

Note: this departs from the usual “all NextBSD changes via PR” flow — direct commits to main per request, to avoid PR round-trips. The one real risk is build-chain ordering: a push triggers that repo’s build, which consumers ingest via the rolling continuous tag. Commit upstream→downstream so nothing rebuilds against a stale upstream.

Order

  1. nextbsd-kernel-toolchain — drop BuildKit GHA cache (§3c); let the image rebuild on 15.1 (this is also the version-fix linchpin).
  2. nextbsd-kernel — drop ccache + repoint (§3a).
  3. nextbsd-freebsd-compat — drop ccache + repoint (§3a).
  4. nextbsd-kernel-modules — drop ccache + repoint (§3a).
  5. nextbsd — drop distfiles cache (§3b).
  6. Purge all five repos’ Actions caches (§4); refresh the GHCR toolchain image.

Each commit message should note “drop CI cache (determinism); see plan” and the --cross-bindir repoint where applicable. After the toolchain rebuilds green, the downstream builds pick up the fresh image with no cache to mask drift.

7. References

Filed 2026-06-21. From a 3-agent investigation of the nextbsd-redux build chain (ccache trio, distfiles, toolchain + purge + root-cause) with the contradictory root-cause claims resolved against the kernel rebrand source. Cache-drop is mechanical; the freebsd-version fix is the toolchain-image refresh + the separately-tracked 15.1 base bump.