nextbsd-userland — a cross-built Apple-userland repo Plan ready

Split NextBSD's Apple-derived userland into its own repo that cross-compiles on a Linux host (like nextbsd-freebsd-compat), publishes a 4th rolling continuous artifact, drops the in-image build toolchain, and turns nextbsd into a thin ISO assembler.

TL;DR

Today nextbsd is a hybrid: it ingests 3 artifacts (compat base, kernel, kexts) but then runs ~2,260 lines of in-chroot Apple-userland compilation using ports cmake/ninja/llvm19. The proposal moves all of that into a new nextbsd-userland repo that cross-builds on Linux with the toolchain image's clang (the same one compat/kernel/modules already use), so nextbsd becomes: download 4 tarballs → layer → uzip → ISO → boot-test.

Feasibility is better than expected. Two facts the agents corrected: (1) only 2 components are actually CMake (libdispatch, swift-foundation-icu) — libCoreFoundation already builds via bsd.lib.mk; (2) the other 18+ components are FreeBSD bsd.lib.mk/bsd.prog.mk Makefiles with no hardcoded compiler/triple — dropped into make.py … buildenv they cross-compile with zero source changes. The image's buildpkgs.txt (cmake/ninja/pkgconf/llvm19) becomes empty; cmake just runs on the runner for the 2 holdouts.

The real work is harness-level, not source: relocate a couple of build-time runs of cross binaries (test_corefoundation, kextdeps) to the boot-test, replace ~26 in-chroot ldd/ldconfig checks with readelf, write 2 CMake cross-toolchain files, and stage the sysroot from compat's continuous base. build.sh shrinks from ~3,245 → ~700 lines.

1. Why do this

2. Feasibility (what the audit found)

The two corrections that de-risk this:

So "drop cmake/ninja from the image" is automatic the moment builds leave the image — the GitHub runner (Ubuntu 24.04) keeps cmake/ninja for the 2 holdouts, cross-compiling them with a toolchain file. The product ships neither.

3. The build chain: 4 repos → 5

Insert nextbsd-userland after compat (it needs compat's base as a link sysroot), parallel to kernel/modules; it becomes the trigger of nextbsd:

releng/15.x (freebsd-src) │ push ▼ nextbsd-kernel-toolchain ──────────── builds the per-arch clang cross image │ dispatch: toolchain-updated ┌───┴────────────────────────┬───────────────────────┐ ▼ ▼ ▼ nextbsd-freebsd-compat nextbsd-kernel (toolchain image = shared dep) (base continuous) (kernel-obj continuous) │ │ dispatch: kernel-updated │ ▼ │ nextbsd-kernel-modules (kexts continuous) │ │ dispatch: base-updated ◄── RE-POINTED from nextbsd to userland ▼ nextbsd-userland ◄── NEW. sysroot staged from compat continuous; (Apple userland continuous) cross-builds the Apple stack; parallel to kernel/modules │ │ dispatch: userland-updated ◄── NEW edge → nextbsd ▼ nextbsd (ASSEMBLER) ── downloads 4 continuous tarballs (base + kernel + kexts + userland) → layer → rootfs → mkuzip → ISO → boot-test
Freshness: every consumer pulls upstreams from rolling continuous (not run-IDs), so nextbsd ingests whatever each continuous currently holds. Userland is the slowest leg (ICU/CF/launchd), so by the time it dispatches nextbsd, kernel+modules+base are virtually always already refreshed — the same eventual-consistency the chain tolerates today (modules already lag kernel). If strict 4-way join is ever needed, gate nextbsd on both userland-updated and kernel-modules with a concurrency group. Recommend eventual.
It's a DAG, not a line. compat is a sibling branch to kernelmodules (both off the toolchain), not downstream of them. nextbsd-userland's only build-time inputs are the toolchain image (cross-clang + cross-tools + the full FreeBSD /usr/include) and compat's continuous base (the built libc/libsys/libs it links against). It does not need kernel or modules to build — libIOKit/kext_tools/DiskArbitration touch the kernel only at runtime and ship their own headers. The 4-way combine (base + kernel + kexts + userland) happens in the nextbsd assembler, not here.

4. nextbsd-userland cross-build harness

Mirror compat's build.yml (per-arch toolchain-image container → reuse the image's baked kernel-toolchain by entering buildenv directly → curated build into /stage via buildenv BUILDENV_SHELL → raw-tar pack → publish job refreshes continuous + cascades). Key points:

Artifact: nextbsd-userland-<arch>.tar.gz — a gzipped tar of /stage rooted at / (so /usr/lib/system/*.so*, /bin, /sbin, /usr/sbin, /usr/lib/pam_*.so.6, /usr/include/*, /usr/tests/freebsd-launchd-mach/*), METALOG and /etc+/var stripped (nextbsd owns config). nextbsd extracts it over the base, userland-over-base (Apple tools overwrite FreeBSD paths — the port-then-drop doctrine).

5. Build order (a real DAG)

TierComponents (in order)Notes
0 — host toolmigcom + mig.shBuilt for the runner (not cross); arch-neutral codegen. Consumed by liblaunch, libdispatch, configd, SystemConfiguration, Libnotify, syslog, IPConfiguration.
1 — foundation libslibmach→libsystem_kernel → libdispatch (cmake) → libxpcliblaunchswift-foundation-icu (cmake) → libCoreFoundationlibmach installs mach/* headers into the sysroot first; dispatch needs migcom (HAVE_MACH); CF needs dispatch+icu+mach.
2 — daemons/serviceslaunchd+launchctlconfigdlibSystemConfigurationlibIOKit+ioregkext_toolsLibnotify+notifydsyslog stack → IPConfigurationmDNSResponderDiskArbitrationhostnamedEach needs Tier-1 libs + its MIG stubs. The kernel-ABI ones (libIOKit /dev/ioregistry, kext_tools /dev/iocatalogue, DiskArbitration) self-SKIP until the matching kernel ingest, same as today.
3 — out of scope (v1)OpenPAM/pam_modules · command suitesPAM deferred to a later pass; the Apple command suites are dropped entirely — see §5a.

LaunchDaemon plists and /etc/asl.conf ship with their daemons from userland (a daemon is useless without its plist — keeping them together prevents drift). ssh-bonjour (the _ssh._tcp register helper) moves to userland; OpenSSH itself stays in the assembler.

5a. Scope decision: the Apple system layer only — POSIX commands come from FreeBSD

nextbsd-userland v1 ships only Tiers 0–2: the Mach/launchd/configd/CoreFoundation/IOKit system layer that has no FreeBSD equivalent. It deliberately does not ship the Apple POSIX command suites (file_cmds, shell_cmds, text_cmds, adv_cmds, system_cmds).

Why drop them: for generic POSIX tools (cat, grep, sed, ls, cp, chmod…) Apple's build buys nothing over FreeBSD's — its only differentiators are Apple-storage facilities NextBSD doesn't have (clonefile/APFS, copyfile, code-signing). Porting them means stubbing out Apple-isms to arrive back at FreeBSD behavior, and it created a ~40-command deferred-shim treadmill (the audit's "Deferred" set). Worse, today the base commands are a transient VM-coreutils copy (build.sh:162-200, comment: the rest are "a file-purity follow-up") that the Apple suites partially overwrite — a known wart.

The fix: get the POSIX userland from FreeBSD source, curated in nextbsd-freebsd-compat and synced to releng/15.1 like the rest of the base. Add the needed FreeBSD bin/, usr.bin/, bin/* dirs (cat, ls, cp, mv, rm, chmod, echo, grep, sed, date, find, getty, … plus md5, stat, etc.) to compat's srclist.txt — they're standard bsd.prog.mk dirs compat already cross-builds trivially, and they track the releng train automatically. This retires the transient-coreutils hack and the Apple command suites in one move, and is the cleaner home: the commands live next to the rest of the FreeBSD base they belong to.

Net effect on the three repos: nextbsd-userland = Apple system layer (Tiers 0–2). nextbsd-freebsd-compat grows the POSIX command dirs in its srclist.txt (synced to releng). nextbsd assembler layers both. getty is the one to watch — FreeBSD getty is more init-coupled (gettytab/ttys) than Apple's launchd-run one, so reverting it needs care, unlike the pure POSIX tools. PAM stays a separate, later decision (it's the Apple login path, not a generic command).

6. nextbsd becomes the assembler

Delete one contiguous block — build.sh:421 (file_cmds) through :2685 (pam_modules), plus OpenSSH/ssh-bonjour :2746-2819: all 29 make -C src/… invocations, the 54 cc -o …/test_*/daemon compiles, the 2 in-chroot cmake -G Ninja heredocs, the ports-llvm19 toolchain shim, the MIG-codegen loops, and every interleaved ldconfig/ldd verify. The src/ tree moves to the new repo.

Keep (the assembler residue): base-tarball extract + /etc//var hand-build; pkg bootstrap + certctl rehash; kernel ingest + kldxref + driver-kext install; overlay apply + pwd_mkdb/cap_mkdb regen; version/os-release stamp; libscan ELF closure; makefs/ESP/mkimg disk image; mkuzip/mfsroot/mkisoimages.sh live ISO. Net: ~3,245 → ~700 lines.

Assembler keeps a runtime dependency on userland binaries. The overlay step invokes Apple's pwd_mkdb/cap_mkdb (from userland) to regenerate /etc/*.db after nextbsd lays its own master.passwd/login.conf — so the userland tarball must be layered in before the overlay step. And nextbsd-version's $IMG_DATE "single source of truth" means the template ships from userland but the @@VERSION@@ stamp stays in the assembler.

buildpkgs.txt / pkglist.txt deltas

buildpkgs.txt (cmake/ninja/pkgconf/llvm19) — all four are consumed only by the in-chroot Apple builds; with those gone, drop all four (keep a comment-only stub for set -u safety). BUILD_PKGS becomes empty, so the toolchain shim + purge blocks become no-ops. pkglist.txt is already fully commented out — no change. (This supersedes the current PR #348, which keeps llvm19 for the in-image build that this plan removes.)

7. The real work: chroot-assumption rework

The component builds don't use the chroot — they're host bmake. The chroot is used only for verification and a few runtime steps. Three classes need rework:

StepAssumptionFixEffort
21 bsd.mk componentshost ${CC} on a FreeBSD VMrun in make.py buildenv → cross-clang; SYSROOT/DESTDIR=/stageLow (no source change)
libdispatch / swift-foundation-icu cmakein-chroot cmake -G Ninja + native clanghost cmake/ninja + a cross-<arch>.cmake toolchain file (CMAKE_C_COMPILER=$CROSS_BINDIR/clang, --sysroot=/stage, CMAKE_INSTALL_LIBDIR=lib/system)Medium (1 file)
test_corefoundation (:1429), kextdeps (:2005)runs a cross-built target binary on the build host — fails under crossmove the runtime assertion to the post-boot test (run.sh already runs there); keep compile+readelf as the build gateLow — highest risk if missed
~26 chroot ldd/ldconfig -m verify/prime sitesrun FreeBSD target binaries in-chrootdrop hint-priming (Lever-B STANDARD_LIBRARY_PATH already resolves /usr/lib/system without hints); replace ldd asserts with readelf -d (DT_NEEDED/RPATH) via cross binutilsLow–medium (mechanical, many sites)
OpenSSH ./configureautoconf native buildcross --host + cache vars — but OpenSSH stays in the assembler, so out of userland scopen/a here

bsd.lib.mk doesn't auto-create INCSDIR//usr/lib/system, so the curated build shell must pre-mkdir every install dir (build.sh already does this per-component — carry those lines into the harness).

8. Phased rollout (keep the green chain intact)

Phase 0 — stand up nextbsd-userland in parallel, non-cascading

New repo = the src/ tree + build-userland.sh (the migrated, cross-shaped §3a3–§3z) + the build.yml. No downstream dispatch; compat still triggers nextbsd. Trigger userland by workflow_dispatch only; iterate the cross-build until it publishes a green amd64 continuous. nextbsd untouched and green throughout.

Phase 1 — dual-source verify behind a flag

Add the 4th download + a NEXTBSD_USERLAND_FROM_ARTIFACT gate to nextbsd/build.sh: when set, skip the §3* builds and tar -xzf the userland artifact instead. Run under workflow_dispatch and compare the boot-test markers (LIBSYSTEM-KERNEL-OK, CONFIGD-*, SC-*, IOKIT-*, HOSTNAMED-*) against a known-good in-chroot build. The /usr/tests/* binaries now arrive inside the tarball, so run.sh works unchanged.

Phase 2 — flip the cascade (one coordinated PR-pair)

Re-point compat's base-updated dispatch from nextbsd to nextbsd-userland; add userland's userland-updatednextbsd; change nextbsd's repository_dispatch.types to [userland-updated]; make the artifact path the default. No window where nothing dispatches nextbsd.

Phase 3 — delete dead weight

Remove §3a3–§3z + OpenSSH-userland glue from build.sh, delete the 4 buildpkgs.txt lines + the toolchain-shim/purge blocks, and move src/ out of nextbsd into nextbsd-userland.

Rollback: until Phase 3 deletes the §3* code, any userland-artifact regression is reverted by clearing the env flag (Phases 1–2) — nextbsd falls back to building in-chroot. The green chain is recoverable at every step before the final delete.

9. Risks

10. References


Filed 2026-06-21. From a 3-agent scope (cross-build harness, migration map, ISO-assembler + CI topology) over nextbsd-work and nextbsd-freebsd-compat. Headline: the 18+ bsd.mk components cross-compile with zero source changes; only 2 CMake holdouts and a handful of chroot-assumption relocations stand between today's in-image build and a clean cross-built userland repo.