Split NextBSD's Apple-derived userland into its own repo that cross-compiles on a Linux host (like nextbsd-freebsd-compat), publishes a 4th rolling continuous artifact, drops the in-image build toolchain, and turns nextbsd into a thin ISO assembler.
Today nextbsd is a hybrid: it ingests 3 artifacts (compat base, kernel, kexts) but then runs ~2,260 lines of in-chroot Apple-userland compilation using ports cmake/ninja/llvm19. The proposal moves all of that into a new nextbsd-userland repo that cross-builds on Linux with the toolchain image's clang (the same one compat/kernel/modules already use), so nextbsd becomes: download 4 tarballs → layer → uzip → ISO → boot-test.
Feasibility is better than expected. Two facts the agents corrected: (1) only 2 components are actually CMake (libdispatch, swift-foundation-icu) — libCoreFoundation already builds via bsd.lib.mk; (2) the other 18+ components are FreeBSD bsd.lib.mk/bsd.prog.mk Makefiles with no hardcoded compiler/triple — dropped into make.py … buildenv they cross-compile with zero source changes. The image's buildpkgs.txt (cmake/ninja/pkgconf/llvm19) becomes empty; cmake just runs on the runner for the 2 holdouts.
The real work is harness-level, not source: relocate a couple of build-time runs of cross binaries (test_corefoundation, kextdeps) to the boot-test, replace ~26 in-chroot ldd/ldconfig checks with readelf, write 2 CMake cross-toolchain files, and stage the sysroot from compat's continuous base. build.sh shrinks from ~3,245 → ~700 lines.
cmake/ninja/pkgconf/llvm19 baked into the image — the whole "ship build deps so it can build Gershwin in place" class (and the stale-build-deps bugs) goes away.continuous release — exactly like kernel/compat/modules. Reproducible, independently testable, cross-built.nextbsd becomes a pure assembler (~700 lines): ingest base + kernel + kexts + userland, layer, package, boot-test. Easy to reason about; fast.libdispatch and swift-foundation-icu. libCoreFoundation ships a vendored CMakeLists.txt but build.sh builds it through its bsd.lib.mk Makefile — already bmake.bsd.lib.mk/bsd.prog.mk (LIB=/PROG=/SRCS=/LIBADD=, inherit ${CC}). In compat's make.py … buildenv they pick up the cross-clang for free. No source changes.So "drop cmake/ninja from the image" is automatic the moment builds leave the image — the GitHub runner (Ubuntu 24.04) keeps cmake/ninja for the 2 holdouts, cross-compiling them with a toolchain file. The product ships neither.
Insert nextbsd-userland after compat (it needs compat's base as a link sysroot), parallel to kernel/modules; it becomes the trigger of nextbsd:
continuous (not run-IDs), so nextbsd ingests whatever each continuous currently holds. Userland is the slowest leg (ICU/CF/launchd), so by the time it dispatches nextbsd, kernel+modules+base are virtually always already refreshed — the same eventual-consistency the chain tolerates today (modules already lag kernel). If strict 4-way join is ever needed, gate nextbsd on both userland-updated and kernel-modules with a concurrency group. Recommend eventual.compat is a sibling branch to kernel→modules (both off the toolchain), not downstream of them. nextbsd-userland's only build-time inputs are the toolchain image (cross-clang + cross-tools + the full FreeBSD /usr/include) and compat's continuous base (the built libc/libsys/libs it links against). It does not need kernel or modules to build — libIOKit/kext_tools/DiskArbitration touch the kernel only at runtime and ship their own headers. The 4-way combine (base + kernel + kexts + userland) happens in the nextbsd assembler, not here.Mirror compat's build.yml (per-arch toolchain-image container → reuse the image's baked kernel-toolchain by entering buildenv directly → curated build into /stage via buildenv BUILDENV_SHELL → raw-tar pack → publish job refreshes continuous + cascades). Key points:
nextbsd-kernel-toolchain image bakes kernel-toolchain (bootstrap-tools + cross-tools + includes) into /usr/obj; nextbsd-kernel reuses it via buildenv and builds fast. Compat copies a slow per-run make.py … toolchain _includes (rebuilds the world toolchain); userland instead enters buildenv against the baked obj. Measured the buildenv-entry step at ~1 s (vs the old rebuild still running past 11 min) — both arches green in ~72 s total, the bulk of which is the one-time container pull. If a world component ever needs build-tools beyond kernel-toolchain, bake the world toolchain into the image (speeds compat too) rather than rebuilding per run.continuous base — gh release download continuous -R …/nextbsd-freebsd-compat --pattern nextbsd-base-<arch>.tar.gz, extract into $SYSROOT. This is a hard dependency: userland can't link libc/libsys/libs without it (measured ~3 s).migcom for the runner (x86_64) before the cross buildenv — it must execute during the cross build (7 components run it on their .defs).chmod u+s on su/login/passwd before tar, per the base-build METALOG convention.Artifact: nextbsd-userland-<arch>.tar.gz — a gzipped tar of /stage rooted at / (so /usr/lib/system/*.so*, /bin, /sbin, /usr/sbin, /usr/lib/pam_*.so.6, /usr/include/*, /usr/tests/freebsd-launchd-mach/*), METALOG and /etc+/var stripped (nextbsd owns config). nextbsd extracts it over the base, userland-over-base (Apple tools overwrite FreeBSD paths — the port-then-drop doctrine).
| Tier | Components (in order) | Notes |
|---|---|---|
| 0 — host tool | migcom + mig.sh | Built for the runner (not cross); arch-neutral codegen. Consumed by liblaunch, libdispatch, configd, SystemConfiguration, Libnotify, syslog, IPConfiguration. |
| 1 — foundation libs | libmach→libsystem_kernel → libdispatch (cmake) → libxpc → liblaunch → swift-foundation-icu (cmake) → libCoreFoundation | libmach installs mach/* headers into the sysroot first; dispatch needs migcom (HAVE_MACH); CF needs dispatch+icu+mach. |
| 2 — daemons/services | launchd+launchctl → configd → libSystemConfiguration → libIOKit+ioreg → kext_tools → Libnotify+notifyd → syslog stack → IPConfiguration → mDNSResponder → DiskArbitration → hostnamed | Each needs Tier-1 libs + its MIG stubs. The kernel-ABI ones (libIOKit /dev/ioregistry, kext_tools /dev/iocatalogue, DiskArbitration) self-SKIP until the matching kernel ingest, same as today. |
| 3 — out of scope (v1) | OpenPAM/pam_modules | PAM deferred to a later pass; the Apple command suites are dropped entirely — see §5a. |
LaunchDaemon plists and /etc/asl.conf ship with their daemons from userland (a daemon is useless without its plist — keeping them together prevents drift). ssh-bonjour (the _ssh._tcp register helper) moves to userland; OpenSSH itself stays in the assembler.
nextbsd-userland v1 ships only Tiers 0–2: the Mach/launchd/configd/CoreFoundation/IOKit system layer that has no FreeBSD equivalent. It deliberately does not ship the Apple POSIX command suites (file_cmds, shell_cmds, text_cmds, adv_cmds, system_cmds).
Why drop them: for generic POSIX tools (cat, grep, sed, ls, cp, chmod…) Apple's build buys nothing over FreeBSD's — its only differentiators are Apple-storage facilities NextBSD doesn't have (clonefile/APFS, copyfile, code-signing). Porting them means stubbing out Apple-isms to arrive back at FreeBSD behavior, and it created a ~40-command deferred-shim treadmill (the audit's "Deferred" set). Worse, today the base commands are a transient VM-coreutils copy (build.sh:162-200, comment: the rest are "a file-purity follow-up") that the Apple suites partially overwrite — a known wart.
nextbsd-freebsd-compat and synced to releng/15.1 like the rest of the base. Add the needed FreeBSD bin/, usr.bin/, bin/* dirs (cat, ls, cp, mv, rm, chmod, echo, grep, sed, date, find, getty, … plus md5, stat, etc.) to compat's srclist.txt — they're standard bsd.prog.mk dirs compat already cross-builds trivially, and they track the releng train automatically. This retires the transient-coreutils hack and the Apple command suites in one move, and is the cleaner home: the commands live next to the rest of the FreeBSD base they belong to.Net effect on the three repos: nextbsd-userland = Apple system layer (Tiers 0–2). nextbsd-freebsd-compat grows the POSIX command dirs in its srclist.txt (synced to releng). nextbsd assembler layers both. getty is the one to watch — FreeBSD getty is more init-coupled (gettytab/ttys) than Apple's launchd-run one, so reverting it needs care, unlike the pure POSIX tools. PAM stays a separate, later decision (it's the Apple login path, not a generic command).
Delete one contiguous block — build.sh:421 (file_cmds) through :2685 (pam_modules), plus OpenSSH/ssh-bonjour :2746-2819: all 29 make -C src/… invocations, the 54 cc -o …/test_*/daemon compiles, the 2 in-chroot cmake -G Ninja heredocs, the ports-llvm19 toolchain shim, the MIG-codegen loops, and every interleaved ldconfig/ldd verify. The src/ tree moves to the new repo.
Keep (the assembler residue): base-tarball extract + /etc//var hand-build; pkg bootstrap + certctl rehash; kernel ingest + kldxref + driver-kext install; overlay apply + pwd_mkdb/cap_mkdb regen; version/os-release stamp; libscan ELF closure; makefs/ESP/mkimg disk image; mkuzip/mfsroot/mkisoimages.sh live ISO. Net: ~3,245 → ~700 lines.
pwd_mkdb/cap_mkdb (from userland) to regenerate /etc/*.db after nextbsd lays its own master.passwd/login.conf — so the userland tarball must be layered in before the overlay step. And nextbsd-version's $IMG_DATE "single source of truth" means the template ships from userland but the @@VERSION@@ stamp stays in the assembler.buildpkgs.txt (cmake/ninja/pkgconf/llvm19) — all four are consumed only by the in-chroot Apple builds; with those gone, drop all four (keep a comment-only stub for set -u safety). BUILD_PKGS becomes empty, so the toolchain shim + purge blocks become no-ops. pkglist.txt is already fully commented out — no change. (This supersedes the current PR #348, which keeps llvm19 for the in-image build that this plan removes.)
The component builds don't use the chroot — they're host bmake. The chroot is used only for verification and a few runtime steps. Three classes need rework:
| Step | Assumption | Fix | Effort |
|---|---|---|---|
| 21 bsd.mk components | host ${CC} on a FreeBSD VM | run in make.py buildenv → cross-clang; SYSROOT/DESTDIR=/stage | Low (no source change) |
libdispatch / swift-foundation-icu cmake | in-chroot cmake -G Ninja + native clang | host cmake/ninja + a cross-<arch>.cmake toolchain file (CMAKE_C_COMPILER=$CROSS_BINDIR/clang, --sysroot=/stage, CMAKE_INSTALL_LIBDIR=lib/system) | Medium (1 file) |
test_corefoundation (:1429), kextdeps (:2005) | runs a cross-built target binary on the build host — fails under cross | move the runtime assertion to the post-boot test (run.sh already runs there); keep compile+readelf as the build gate | Low — highest risk if missed |
~26 chroot ldd/ldconfig -m verify/prime sites | run FreeBSD target binaries in-chroot | drop hint-priming (Lever-B STANDARD_LIBRARY_PATH already resolves /usr/lib/system without hints); replace ldd asserts with readelf -d (DT_NEEDED/RPATH) via cross binutils | Low–medium (mechanical, many sites) |
OpenSSH ./configure | autoconf native build | cross --host + cache vars — but OpenSSH stays in the assembler, so out of userland scope | n/a here |
bsd.lib.mk doesn't auto-create INCSDIR//usr/lib/system, so the curated build shell must pre-mkdir every install dir (build.sh already does this per-component — carry those lines into the harness).
New repo = the src/ tree + build-userland.sh (the migrated, cross-shaped §3a3–§3z) + the build.yml. No downstream dispatch; compat still triggers nextbsd. Trigger userland by workflow_dispatch only; iterate the cross-build until it publishes a green amd64 continuous. nextbsd untouched and green throughout.
Add the 4th download + a NEXTBSD_USERLAND_FROM_ARTIFACT gate to nextbsd/build.sh: when set, skip the §3* builds and tar -xzf the userland artifact instead. Run under workflow_dispatch and compare the boot-test markers (LIBSYSTEM-KERNEL-OK, CONFIGD-*, SC-*, IOKIT-*, HOSTNAMED-*) against a known-good in-chroot build. The /usr/tests/* binaries now arrive inside the tarball, so run.sh works unchanged.
Re-point compat's base-updated dispatch from nextbsd to nextbsd-userland; add userland's userland-updated → nextbsd; change nextbsd's repository_dispatch.types to [userland-updated]; make the artifact path the default. No window where nothing dispatches nextbsd.
Remove §3a3–§3z + OpenSSH-userland glue from build.sh, delete the 4 buildpkgs.txt lines + the toolchain-shim/purge blocks, and move src/ out of nextbsd into nextbsd-userland.
nextbsd falls back to building in-chroot. The green chain is recoverable at every step before the final delete.test_corefoundation, kextdeps) — must relocate to boot-test or they fail the instant the host can't exec a cross binary.continuous base — the dispatch ordering (compat → userland) is load-bearing, not a nicety. (The artifact already exists today, so no new bootstrap.).incbin restructure).nextbsd-freebsd-compat/.github/workflows/build.yml (CI template + the base-updated dispatch to re-point), nextbsd/build.sh (delete §3a3–§3z; keep ingest+assembly), nextbsd/buildpkgs.txt, nextbsd/src/ (the tree that migrates).Filed 2026-06-21. From a 3-agent scope (cross-build harness, migration map, ISO-assembler + CI topology) over nextbsd-work and nextbsd-freebsd-compat. Headline: the 18+ bsd.mk components cross-compile with zero source changes; only 2 CMake holdouts and a handful of chroot-assumption relocations stand between today's in-image build and a clean cross-built userland repo.