NextBSD Driver Delivery — kld / kext / IOKit scoping

Draft · sidebar to the monorepo plan This is the IOKit/kext breakout called out as unsettled in the monorepo plan's §12. It builds on your extensive existing work (the ko→kext conversion plan, the kld-to-kext catalog/naming, the in-kernel matcher feasibility, the nvidia kext + userland plans, the graphics plan, the input spike) rather than restarting the question. Where it disagrees with an older draft, it says so.

Verdict

No IOKit runtime None of your four goals needs Apple's in-kernel IOKit C++ runtime (OSMetaClass / IOService / IORegistry / libkern C++). Your own shipped design already agrees — the matcher you built is flat C on newbus, not OSDictionary, and your ko→kext plan scoped the real runtime out as a "3–5 engineer-year diversion that collides with newbus." "kext" = a bundle wrapping a verbatim .ko that kld loads — that is entirely separable from the IOKit runtime, and you keep 100% of it.
A mixture, not "everything a kext" Three tiers: compiled-in (boot minimum) / plain kld (devmatch-autoloaded: input, common ethernet, most drivers) / kext (firmware ∨ multi-version). The rule: kext ⇐ has-firmware ∨ needs-multi-version. Of your catalog's 343 "kext candidates," only ~50–60 actually meet that bar — the other ~250 should stay plain klds that native devmatch already matches, shrinking what your personality generators + in-kernel matcher must cover.
Firmware offline = solved, pkg-free The runtime firmware path is 100% local; fwget/pkg are only an install-time vehicle. FreeBSD 15.0 added raw-file firmware from /boot/firmware/ — bake blobs into the image, done. Your in-kext Resources/firmware/ bundling (already shipping) is a legitimate curation layer on top, not what makes offline work.
Wired-memory win is modest Honest number: tens of MB, not hundreds — a stripped 15 MB kernel showed the same 120 MB wired as GENERIC because the RAM-scaled vm_page array (~42 MB) dominates. The big consumers (nvidia, amdgpu, zfs) are already loadable in stock FreeBSD. So lead the kext case with version-coexistence + the IOKit-shaped registry + modularity/attack-surface, not raw MB.
nvidia coexistence: on-disk only Install 470/535/595 side-by-side and auto-select per GPU — yes, no pkg, no IOKit. Two branches resident at once for two GPUs — no, on any OS (singleton nvidia module name → EEXIST). IOKit wouldn't fix it either. This is a documented wall, not a defect.
One build system make buildkernel already emits the klds. Add a NextBSD-owned make extensions target (formalizing ko2kext.sh) that wraps the built .kos into kexts — zero edits to stock FreeBSD build files.

0 · What's already decided 1 · The IOKit verdict (A/B/C) 2 · Your four reasons, honestly 3 · The three-tier form policy 4 · Matching: devmatch + your matcher 5 · Firmware without pkg 6 · Multi-version coexistence 7 · One build system (make extensions) 8 · Open items & honest tensions 9 · Decisions for you

0What's already decided (don't relitigate)

The recon read all ~14 of your planning docs plus the live ko2kext.sh + gen-*-personalities.sh tooling and CI. You've made and shipped more of this than the "should we do IOKit" framing implied. This doc treats the following as settled and builds on them.

A kext wraps a verbatim, unmodified FreeBSD .ko; ELF stays ELF, kld is the real link/load engine, and kextload/stat/unload are thin front-ends over kldload/….
Apple's in-kernel IOKit C++ runtime (OSMetaClass/OSObject/IOService RTTI/KXLD) is OUT — ruled out consistently for three months as a 3–5 engineer-year diversion.
Device→driver matching runs in-kernel via a flat-C IOCatalogue + matcher on newbus's device_nomatch path, reading IOPCIPrimaryMatch/IOProbeScore from bundle personalities — built and hardware-verified on a T420 (0x24f38086 → IntelWiFi).
Personalities are auto-generated from each driver's own device-id tables (the six gen-*-personalities.sh), never hand-curated — the hand-curated experiment was explicitly ripped out as unscalable.
Firmware is bundled inside the kext (Contents/Resources/firmware/, symlinks dereferenced) and found by firmware(9) — offline, no pkg. Already implemented.
Naming: org.nextbsd.{driver,filesystems,crypto,net,security,kpi,compat,firmware,kext}.*; CFBundleExecutable = the real KMOD= name; nvidia = NVIDIAGraphics<NNN> per branch.
Base-owned, never pkg — kexts ride the NextBSD-kernel-extensions package into /System/Library/Extensions; nvidia userland is staged from the vendor tarball with an LLVM-free libgbm closure guard (to stay off mesa→llvm19).

The one thing NOT to relitigate Whether to implement Apple's in-kernel IOKit C++ runtime. Settled NO across every doc, and the shipped design is unambiguous. Any "IOKit" in NextBSD is the flat-C matcher bolted onto newbus — already built — not the object runtime. §1 confirms this holds under fresh analysis.

1The IOKit verdict — A, B, or C

The vocabulary is the trap: in NextBSD, "kext" = delivery/matching/versioning wrapper over a kld. That is separable from the IOKit runtime. You can keep the entire kext model and never link a line of libkern C++.

Option	What	Assessment
A	Native `kld` + `devmatch` (and your thin flat-C matcher) + personality/selector veneer + firmware-in-`.ko`/bundle. No IOKit runtime.	Recommended — and what you already built. Delivers all four goals. You keep the Apple-shaped bundle, `/System/Library/Extensions`, `IOKitPersonalities`, Apple-named CLIs, the in-kernel matcher, and the `HOST_KEXTD_PORT` load channel — all on unmodified `kld`.
B	A + a minimal source-level IOKit-compat layer for one subsystem (HID or a graphics family), to reuse Apple family source.	No taker. You use `evdev`/`hid` and `drm-kmod`, not Apple family source — so it would cost a libkern-C++/OSObject subsystem (the very wired memory goal 4 forbids) for a capability nothing consumes. Skip.
C	Full in-kernel IOKit runtime — OSMetaClass, IOService, libkern C++, IORegistry object graph, real OSKext Mach-O linking.	Never justified for these goals. 3–5 engineer-years, duplicates newbus (two probe/attach engines fighting over `device_t`), and regresses goals 1 and 4. Only a hard pivot to reusing Apple kernel driver source verbatim would justify it — that's ravynOS abandoning the FreeBSD kernel for XNU, a different project.

Why the graphics case refutes rather than proves the need: GPUs sound like where IOKit earns its keep, but FreeBSD already runs modern Intel/AMD/Radeon — full KMS, /dev/dri, the hardest driver class — through drm-kmod + LinuxKPI with zero IOKit, and your nextbsd-graphics-plan does exactly that: IOGraphics.kext is drm.ko renamed. For nvidia the KMS bridge is nvidia-drm bound to drmn via MODULE_DEPEND — a LinuxKPI/newbus story. The hardest remaining piece (Wayland) is a userland GBM backend port, orthogonal to any kernel driver model.

2Your four reasons, mapped honestly

You gave four motivations for the kext model. All are legitimate — here's the lightest mechanism for each, and where IOKit is load-bearing (spoiler: nowhere).

Goal	Lightest mechanism	IOKit runtime needed?
1. Offline firmware, no fwget/pkg	`firmware(9)`: blob in the image (raw `/boot/firmware/` on FreeBSD 15, or `.incbin`'d `.ko`), resolved by name at attach.	No Apple's `OSKextRequestResource` is worse here — async, daemon-served, not on the early-boot path.
2. HW→driver+firmware matching	`devmatch` + `MODULE_PNP_INFO`, or your flat-C matcher reading auto-derived personalities. Same match key (`0xDDDDVVVV`).	No Familiar vocabulary, not a better engine. Your own history proves it: the literal IOKit/`IOPCIMatch` port was ripped out for devmatch because plists "don't scale."
3. nvidia multi-version coexistence	Versioned bundles inert on disk + a selector that autoloads exactly one by matched GPU.	No On-disk coexistence is packaging. The hard limit (one `nvidia`-named module resident) is a linker-namespace wall IOKit doesn't escape; your design makes `OSBundleLibraries` decorative and keeps `MODULE_DEPEND` authoritative.
4. On-demand load, cut wired memory	`kld` autoload on match; boot-critical drivers stay compiled-in/preloaded.	No — runtime is a NET COST An in-kernel IOKit permanently wires the class registry, per-node `OSDictionary`, the full `IOCatalogue`, and `IOWorkLoop` threads — resident regardless of load. That directly contradicts goal 4 (your K2 dodged it on purpose: "flat C struct, not an OSDictionary").

Honest correction on goal 4 The wired-memory payoff is real but modest — tens of MB, not hundreds. Measured: on FreeBSD 14.3 a stripped 15 MB kernel showed the same 120 MB wired as GENERIC, because the vm_page array (~42 MB, scales with installed RAM) + UMA/buffer overhead dominate; driver text is a minority. Input+ethernet together are only a few MB. The genuinely large drivers (nvidia ~30–40 MB, amdgpu+fw ~15–30 MB, zfs ~7 MB) are already loadable in stock FreeBSD — you get those savings without any new tiering. And idle drivers can't be unloaded (attached device → EBUSY; devmatch never unloads), so the lever is "never load the unused driver," not "unload after idle." Meaningful on 256 MB–1 GB embedded targets; marginal on a 16 GB desktop. Lead the case with version-coexistence, the IOKit-shaped registry, and modularity/attack-surface instead.

3The three-tier form policy

Your questions — everything a kext? only firmware? a mixture? drop redundant klds? — answered. The answer is a mixture with an explicit rule, and it sharpens your existing 343-vs-131 split.

All three forms run the same unmodified .ko through the same kld linker. The choice is about packaging cost, so pay a cost only when the driver has the matching need:

P1 boot-essential (root FS / boot console / boot bus)? ── YES ─▶ COMPILED-IN (or loader-preloaded) │ NO P2 has-firmware? OR P3 needs-multi-version? ── YES ─▶ KEXT (personality + firmware + version select) │ NO everything else ─────────▶ PLAIN KLD (devmatch autoload; NO personality)

Class	Form	Deciding property
Root/boot storage (`ahci`, `nvme`, `virtio_blk`, cam), root FS (`ufs`), core buses (`pci`/`acpi`/`cpu`), console (`vt`), USB core+HCs	compiled-in (root HBA + root FS may be loader-preloaded)	P1 — needed before `mountroot` / devmatch
Ethernet (common: `em`/`igb`, `re`, `bge`, `virtio_net`)	plain kld	no firmware, no versioning → devmatch autoloads. The cleanest "not a kext" class.
Input / HID (`hidbus`, `hkbd`/`ukbd`, `hms`/`ums`, touch); small in-base HID leaves	plain kld (tiny leaves already compiled-in per your input spike)	hotplug, no firmware
Audio, bluetooth (unless fw), pseudo-devices, `linuxkpi` shim	plain kld	none of P1–P3 → default tier
Wi-Fi (`iwlwifi`, `rtw88/89`, `ath`, `mt76`)	kext	P2 — essentially every part needs a firmware blob
Firmware ethernet (`ice`+DDP, `cxgbe`, `mxge`)	kext	P2 — firmware
GPU/DRM (`amdgpu`, `i915`, `radeonkms`)	kext	P2 firmware + P3 (drm 6.6/6.12 coexistence)
nvidia (`NVIDIAGraphics<NNN>` per branch)	kext	P3 multi-version (primary) + P2 (GSP fw on modern) — the canonical versioning case

The four sub-questions, answered

Everything a kext? No. For a driver with no firmware and one version (all common ethernet/input/audio/pseudo — the majority), a kext forces a personality that is a lossier re-encoding of the MODULE_PNP_INFO the module already carries, an empty Resources/, and version logic for one version. The only gain is cosmetic uniformity — recoverable more cheaply (§4).
Only firmware drivers? Right instinct, necessary-but-not-sufficient. "Has firmware" is a clean primary trigger (Wi-Fi, modern GPU, ice/cxgbe) — but it under-covers nvidia, whose defining need is versioning, not firmware. So the rule is the union: kext ⇐ has-firmware ∨ needs-multi-version.
A mixture? Yes — this is the answer. The two cost axes (resident-memory vs bundle-machinery) are independent, so you genuinely need three cells. Tighten tier-1 to "the transport this image actually boots on + buses + console" (a product kernel isn't GENERIC install media); make plain-kld the largest tier.
Drop klds where a kext supersedes? Yes — one form per device. If a driver graduates to a kext, its plain .ko must not also sit in /boot/kernel where devmatch could race the kext matcher for the same PCI ID. Enforce structurally: add a form ∈ {builtin, kld, kext, excluded} column to your catalog as the single source of truth, generate build-excludes from it, and let kldxref index only the loose tier-2 .kos — so a kext'd driver physically can't be devmatch-autoloaded.

The biggest refinement to your existing plan Your catalog splits 474 modules into 343 kext candidates vs 131 compiled-in. The analysis says only ~50–60 of the 343 should be actual kexts (the firmware ∨ multi-version subset ≈ ~35 Wi-Fi + ~13 firmware NIC/storage + GPU + nvidia). The other ~250 should stay plain klds on native devmatch — no personality generation, no bundle. That confines every piece of bespoke kext machinery to the drivers that actually need it, and shrinks your open D1 (naming) decision to the ~50–60 real kexts instead of 343.

4Matching: devmatch + your in-kernel matcher are complementary

A genuine tension the research surfaced — handled by scoping, not by tearing anything out.

Native devmatch (reads MODULE_PNP_INFO → linker.hints, autoloads the matching .ko on a newbus NOMATCH event) already does device→driver matching for the whole plain-kld tier — including firmware drivers like iwlwifi/i915, which ship PNP tables too. It's free, authoritative, and handles PCI/USB/ACPI subtleties. So a personality generated for a plain kld would be a lossier copy fighting the real matcher.

Your in-kernel flat-C matcher (built, T420-verified) isn't redundant with devmatch — it serves what devmatch doesn't: feeding the kext autoload channel (HOST_KEXTD_PORT) and populating the IOKit-shaped registry that Gershwin/loginwindow read (the libIOKit facade, HardwareMatch). The clean division:

Plain-kld tier (~250): native devmatch does the matching + autoload. Generate no personalities.
Kext tier (~50–60): your personalities + in-kernel matcher drive kext selection/autoload and the version arbitration devmatch can't do — plus they feed the registry facade.

Net: keep your matcher — but point it at the ~50–60 kexts, and let devmatch carry the long tail. That's less personality-generation and less parallel-matcher surface than the current 343-wide plan, with no loss of capability.

5Firmware without pkg

The offline/no-pkg goal is already met by firmware(9); the kext bundling is curation on top.

The runtime firmware path is 100% local: a driver calls firmware_get("iwlwifi-cc-a0-77.ucode"); the kernel autoloads a firmware .ko that firmware_register()s the blob, or (new in FreeBSD 15.0) reads a raw file from /boot/firmware/. fwget/pkg are only an install-time delivery vehicle — never in the load path. So the no-pkg/offline goal is met simply by baking the blobs into the image.

Preferred target (15.0): raw blobs in /boot/firmware/ — dependency-free, no .ko build. Confirm NextBSD's kernel has the 15.0 raw-file fallback; it's the load-bearing feature.
Your kext bundling (Resources/firmware/, symlinks dereferenced) is a legitimate curation/version-coupling layer — one self-contained unit per device, firmware version bound to driver version. But it doesn't change the load mechanism (the kext must still drop blobs where firmware(9) looks, or register them by name) and it's not what enables offline. Keep it for curation; don't believe it's the enabler.
LinuxKPI name-mangling gotcha: Linux drivers request amdgpu/navi10_*.bin; module names can't contain /, so LinuxKPI tries ~4 name variants. Register/name each unit to match one exactly (safest: basename in /boot/firmware/).
Bloat clarification: firmware packages are dependency-free — the llvm-via-mesa bloat is a separate userland graphics problem. Don't conflate them. (Your nvidia userland plan already keeps libgbm LLVM-free for exactly this.)
Licensing: ship only the WHENCE-redistributable subset and bake the LICENSE/WHENCE files in — you redistribute an installable image, so this matters.

6Multi-version coexistence — the honest split

The case that made you want a kext model. It works — for the coexistence you can actually have.

Case	Feasible?	Mechanism
Sequential-per-boot — install 470/535/595 side-by-side, one loaded, auto-selected by the detected GPU	Yes — straightforward, no pkg, no IOKit	Versioned `.ko` paths (your `NVIDIAGraphics<NNN>.kext`) + a PCI-ID selector reading `supported-gpus.json` `legacybranch` → `kldload` the chosen branch; version-locked userland via the sibling `NVIDIA<NNN>.bundle` + symlink activation (your `nvidia-activate.sh`).
Simultaneous — two branches resident at once for two different-generation GPUs	No — impossible on any OS	All branches compile to the module name `nvidia` → the kernel linker permits one resident (`EEXIST`); one global `/dev/nvidiactl` + userland ABI. IOKit/DKMS/nvidia's own installer can't do it either. Real answers are external (one covering branch, or per-GPU VM passthrough). A documented wall, not a defect of your model.

Because pkg keys one version per origin and collides on shared paths, packages fundamentally can't deliver even the sequential case — which is exactly why your base-owned versioned-bundle model is the right call here. Just document the simultaneous-two-GPU limit in the selector (pick the newest branch covering the primary GPU; warn on an un-coverable secondary).

7One build system — `make extensions`

"Does buildkernel make klds? Do we need a make extensions target?" Yes, and yes — cleanly.

make buildkernel KERNCONF=NEXTBSD already builds the klds. Its all target = kernel-all + modules-all (per sys/conf/kern.post.mk), recursing into sys/modules; installkernel stages kernel + .kos into /boot/kernel. Tier-2 klds are already a native, meta-mode-incremental output. Nothing to add.

Kexts = built .ko + generated personalities + bundled firmware + OSBundleLibraries — exactly what ko2kext.sh + gen-*-personalities.sh already do as a post-build step. Formalize that into a new NextBSD-owned target:

# nextbsd/extensions/Makefile — invoked after buildkernel, keyed off the .ko objdir
extensions: .PHONY
.for k in ${KEXTS}                       # IntelWiFi i915 amdgpu NVIDIAGraphics595 ...
	@${MAKE} -f ${SRCTOP}/nextbsd/share/mk/bsd.kext.mk extension \
	    KEXT_NAME=${${k}_NAME} KEXT_ID=${${k}_ID} KO=${KODIR}/${${k}_KO} \
	    KEXT_PERSGEN=${SRCTOP}/nextbsd/tools/gen-${${k}_GEN}-personalities.sh \
	    KEXT_FW="${${k}_FW}" KEXT_DEPS="${${k}_DEPS}"
.endfor
installextensions: .PHONY
	cp -R ${KEXTDIR}/*.kext ${DESTDIR}/System/Library/Extensions/

Recommendation: a standalone make extensions target (+ nextbsd/share/mk/bsd.kext.mk) that consumes buildkernel's .kos. For NextBSD-original drivers, an opt-in .include <bsd.kext.mk> in their own Makefile emits .ko+bundle in one shot. Reject folding it into make packages — base must not require pkg (pkgbase can wrap the staged bundles later, but assembly stays pkg-free).
Incremental: key each .kext off its .ko mtime — rebuild the bundle whenever the .ko (or the personality generator / firmware set) changes. Assembly is cheap, so rebuild-on-.ko-change is safe.
Three tiers, one pass: compiled-in (kernel image) + plain klds (/boot/kernel/*.ko, stock loading) + kexts (/System/Library/Extensions, separate namespace, no /boot/kernel collisions) all emit from one buildkernel + one extensions pass.
Zero upstream edits: NextBSD-original klds via LOCAL_MODULES; the extensions target in a new nextbsd/ dir invoked from the release/CI driver; a new sys/*/conf/NEXTBSD that include GENERIC — never edit Makefile.inc1, kern.post.mk, kmod.mk, or GENERIC. Move ko2kext.sh + gen-*-personalities.sh verbatim from nextbsd-kernel-modules into nextbsd/tools/.

8Open items & honest tensions

What the recon flags as genuinely unresolved (fair game), plus the tensions this analysis raises.

Autoload bind is not wired end-to-end. The kernel→kextd Mach-send channel (K3b, HOST_KEXTD_PORT) is designed but unbuilt/untested on hardware; depends on EVFILT_MACHPORT (#168) + kernel-native Mach match-notifications (Phase 2), neither implemented. This is the critical path to "kext autoloads on match."
ko2kext.sh can't yet package a multi-.ko kext with per-tier CFBundleExecutable — net-new work needed for the nvidia modeset/core tiers (as written it would fail to load KMS on most branches).
The ~250-plain-kld refinement (§3/§4) is this doc's recommendation against the current 343-wide plan — it needs your buy-in, since it changes what the generators + matcher target.
Display-name style (D1): Apple CamelCase (IntelE1000.kext) vs FreeBSD (if_em.kext) vs hybrid — now scoped to only the ~50–60 kexts.
Firmware runtime-discovery (D4 sub-choice): nest the firmware .ko so firmware_get() is unchanged, vs point firmware(9) at Resources/ (purer, needs a loader hook). The FreeBSD-15 raw /boot/firmware/ path may make this moot.
nvidia specifics: legacy 340/304 may not compile on 15.x (likely 4–5 branches ship); linker.hints cross-branch contamination hazard (multiple nvidia providers on disk); Wayland blocked on the GBM backend.
Verify the numbers on real hardware: the wired-memory figures are FreeBSD-forum-measured + estimates — run kldstat -h / vmstat -z / ls -la /boot/kernel/*.ko on the thinkpad-t460s to replace estimates with your actual driver set.

9Decisions for you

What I'd lock (with your nod)

Option A confirmed — keep the full kext delivery/matching/versioning model; no IOKit C++ runtime (you'd already scoped C out — this just makes it explicit as the answer to the sidebar).
Adopt the three-tier policy + the kext ⇐ firmware ∨ multi-version rule, and add the form column to the catalog as single source of truth with generated build-excludes (one form per device).
Refine 343 → ~50–60 real kexts; let devmatch carry the ~250 plain klds; point your in-kernel matcher + personalities at the kext tier only.
Formalize make extensions into the monorepo, ko2kext moved in, zero upstream edits.

Still genuinely open — worth a sidebar of their own

The K3b autoload channel (Mach-send kernel→kextd) — the real "does a kext autoload on match" gate. Needs a PoC. Want this as its own breakout?
Do you accept the 343→~50–60 narrowing, or is there a reason (e.g. the IOKit-registry facade wanting every device as a personality) to keep more drivers kext-wrapped? This is the one place my recommendation pushes against your current plan — your call.
Multi-.ko kext packaging for nvidia tiers — a concrete ko2kext.sh enhancement to spec.

Every load-bearing claim traces to a source-cited research pass (FreeBSD firmware(9)/devmatch(8)/build(7) + real freebsd-src file:line, Apple IOKit/OSKext + Darling's userspace-only IOKit, nvidia/DKMS multi-version, and your own 14 planning docs + ko2kext tooling read directly). Provisional sidebar to the monorepo plan — IOKit remains your explicitly open topic. No code written, no repos changed in producing this.