Seven kernel subsystems ship 27,455 lines of generated C that nobody can regenerate, and roughly eighteen libmach entry points return KERN_SUCCESS without doing anything. This is the plan to fix the cause rather than the symptoms — ordered so it can be executed straight through.
Kernel MIG works on NextBSD today. Two hand-built messages on real hardware returned correct data, including a complex reply carrying out-of-line memory. No kernel change is required for correctness.
The whole defect is a missing source of truth. Seven kernel MIG servers were generated in 2015 by a compiler we no longer have and checked in as source, with no .defs beside them. Userland then papered over the gap with stubs that fabricate success.
mach_port is reconstructed and verified. Dispatch table 36/36 identical to the frozen server; generated client and generated server agree 35/35 — MIG’s guarantee, restored by construction.
MIG also relieves the syscall-slot pressure that forced us to patch FreeBSD’s lkmnosys band from 10 slots to 58. MIG costs zero slots; the trap approach would need ~156.
Nothing downstream is trustworthy until CI actually gates (#81). That lands first.
Compiled natively on gershwin-on-nextbsd, sent to mach_task_self(). Both messages hand-built to the shape the frozen server demands — MACH_MSGH_BITS_COMPLEX set with a descriptor count of zero.
probe 1 mach_port_get_attributes (3217)
reply 3317 RetCode=0 outCnt=10
mps_pset=0 seqno=0 mscount=0 qlimit=5 MSGCOUNT=0 <-- qlimit=5 is MACH_PORT_QLIMIT_DEFAULT:
real kernel state, not a zeroed buffer
probe 2 mach_port_get_set_status (3214) <-- the exact call libmach fakes
reply 3314 size=56 complex=yes desc_count=1
membersCnt=1 ool.size=4 ool.address=0x1000
member[0] = 0x12 <-- the port we had just insertedProbe 2 exercises vm_map_copyin in the kernel and the OOL copyout into our address space. The kernel implementation was complete all along; userland simply never asked it anything.
mach_port.defs was rebuilt from the frozen server’s dispatch table, using Apple’s open-source declarations for signatures, and run through migcom built from our own tree (bootstrap_cmds-138-freebsd).
| Check | Result |
|---|---|
| Subsystem base / end | 3200 / 3236 — identical |
Slot count, including the skip; tombstone at 3225 | 36 / 36 — identical |
| Dispatch entries (routine name and argument count) | 36 / 36 identical |
| Wire structs vs frozen server | 4 identical, 31 differ only by msgh_body, 0 unexplained |
| Generated server vs generated client | 35 / 35 exact |
That last row is the point of the exercise. Every msgh_id is preserved, and the two ends agree because they came from one description.
This was scoped as one stubbed routine. It is seven subsystems.
| Generated server (kernel, checked in) | Lines | Base | Slots | Reconstructs today? |
|---|---|---|---|---|
mach_port_server.c | 5,720 | 3200 | 36 | yes — verified |
task_server.c | 6,376 | 3400 | 42 | needs mach_zone_name_array_t |
vm_map_server.c | 4,444 | 3800 | 31 | needs vm_map_t + macro-wrapped names |
host_priv_server.c | 4,022 | 400 | 26 | needs vm_map_t |
mach_host_server.c | 3,199 | 200 | 25 | needs clock_id_t |
mach_vm_server.c | 3,028 | 4800 | 20 | needs vm_map_t + macro-wrapped names |
clock_server.c | 666 | 1000 | 3 | needs clock_serv_t |
| total | 27,455 | 183 |
.defs. The six remaining failures are shallow: a missing type vocabulary (the same gap that required writing mach_debug_types.defs from scratch) and routine names wrapped in PREFIX(KERNEL_SERVER_SUFFIX(…)) macros that only resolve after the C preprocessor runs. mach_port was simply the one subsystem without that pattern.
The kernel carries a FreeBSD patch, patches/0001-NextBSD-widen-lkmnosys-dynamic-syscall-band-to-58-sl.patch, whose own commit message reads:
“Append 48 lkmnosys slots (599–646)… lkmnosys slots (210-219) are exhausted by the Mach trap set.”
Stock FreeBSD offers 10 dynamic syscall slots. We widened it to 58. About 14 are in use.
Zero slots, forever. Every routine rides mach_msg_trap — one already-wired syscall. Restoring all seven subsystems (~156 routines) consumes nothing from the band.
One slot per routine. 156 routines would need 156 slots — nearly triple the widened band and 15× stock FreeBSD. It does not scale, and never could.
So the 10→58 widening was not a fix; it was a symptom of an approach with no ceiling. This also retroactively settles closing kernel #123: it would have spent a scarce slot on something Apple does over MIG.
The traps mostly stay, though. Checked against the 130 entries in XNU’s mach_trap_table, 9 of our 12 match a real Apple trap and should remain — reshaped into Apple’s layering, where the trap is tried first and falls back to MIG:
rv = _kernelrpc_mach_port_allocate_trap(task, right, name);
if (rv == MACH_SEND_INVALID_DEST)
rv = _kernelrpc_mach_port_allocate(task, right, name); /* MIG */The three with no Apple counterpart — host_set_special_port, task_get_special_port, task_set_special_port — convert to MIG and free slots. Long term this could shrink or retire patch 0001 entirely: one fewer FreeBSD divergence to carry.
.defs, and what gets committedApple settles both questions. XNU owns osfmk/mach/*.defs and installs them — 28 ship in the macOS SDK under /usr/include/mach/, mach_port.defs among them — and Libsyscall generates its clients from that single copy.
| Committed to git? | Where | |
|---|---|---|
*.defs (hand-authored source) | yes | kernel repo, for the 7 kernel interfaces; installed to the sysroot for userland to build against |
*_server.c / *User.c (generated) | no | regenerated every build; already covered by .gitignore |
nextbsd-userland already does this correctly — 24 .defs committed, 0 generated files committed, and .gitignore lists *Server.c, *User.c, .mig/, *-mig/. Those 24 stay exactly where they are: they describe userland↔userland protocols (configd, launchd, notifyd, ASL, firehose, WLAN) with no kernel side at all.
nextbsd-kernel is the anomaly: 7 committed generated servers, 0 .defs. Only the seven kernel-interface descriptions move there.
mach_types.h, port.h, message.h, and those are deliberately different views — the kernel’s port.h is 451 lines against userland’s 98, carrying internal types. That is correct and matches Apple. But a .defs describes the wire, and two descriptions of one wire is precisely the failure this whole effort exists to remove.
/var/log/asl was never rotated.
Until the boot test fails when the system is broken, every result below — including the MIG work — is unverified by construction. #81 lands first. Everything else inherits its trustworthiness from it.
Fix #81: the boot test must assert that daemons actually came up, not merely that login: appeared. Then land the four green, verified PRs behind it — #76 (the enumeration root cause, verified before/after on hardware), #77 (restore Apple’s boot sequence, closes #70), then #73 and #74.
mach_port.defs done — gate passed36/36 dispatch, 35/35 client↔server. Also produced mach_debug_types.defs, which the tree’s own ipc_info.h has referred to for years without it ever existing.
migcom into the buildCommit mach_port.defs + mach_debug_types.defs to src-overlay/, build migcom as a bootstrap host tool, generate mach_port_server.c at build time, delete the checked-in copy, and install the .defs into the sysroot. Merges before any userland work depends on it.
vm_deallocate in libmach is a no-op. The moment mach_port_get_set_status returns real out-of-line memory, mportset_callback leaks a page per demand-port event — on the hot path. It cannot simply be un-no-op’d: 12+ call sites in libsystem_asl depend on current semantics, and commit 9fe3d79 documents why naïve munmap fails on an interior pointer. That commit also names the fix: an OOL-receive path that hands back a true base and mapped size.
Generate mach_portUser.c; replace the fabricating stubs. On-demand launch is broken twice over and both breaks are in the same loop — get_set_status returns zero members and get_attributes returns KERN_RESOURCE_SHORTAGE, so fixing either alone changes nothing. Closes #79, unblocks #80.
Reshape the 9 surviving traps to trap-first / MIG-fallback; convert the 3 non-Apple traps to MIG. Today resolve_syscall() failure returns KERN_RESOURCE_SHORTAGE; that is exactly where control should fall through to MIG.
#78 (deadlock at boot in notify_register_check — untimed Mach RPC) and #80 (aslmanager never runs). #80 is downstream of on-demand launch; #78 is independent and can run in parallel.
task, vm_map, host_priv, mach_host, mach_vm, clock — one PR each, same method, same gate. Sequenced last because each is mechanical once Phase 2 proves the build wiring.
| Item | Disposition |
|---|---|
| #81 CI boot test non-gating | Phase 0 — promote to blocker |
| #79 on-demand Mach launch dead | closed by Phase 4; update — it is two stubs, not one |
| #78 syslogd boot deadlock | Phase 6 |
| #80 aslmanager never runs | Phase 6, downstream of Phase 4 |
| #70 two concurrent boot loaders | closed by PR #77 |
| PRs #76, #77, #73, #74 | green + verified — merge in Phase 0 order |
| kernel #124 (skill) | revise then merge — it teaches trap-wiring as the default; it must teach MIG-first with traps as an accelerator |
migcom into the kernel build; commit .defs, drop generated C, install to sysrootvm_deallocate is a no-op (blocks #79’s fix from being leak-free)mach_port_mod_refs real is a behavioural change, not a bug fix. It currently no-ops. Once it actually drops references, any latent over-release or missing-retain in the tree stops being harmless and starts destroying ports. Land it in its own commit, separable from the rest.
msgh_id drift. IDs are base + index. One misplaced skip; silently renumbers every routine after it, and the failure mode is calling the wrong kernel function rather than an error. The dispatch-table diff is the only thing that catches this; it is not optional for any of the seven.
Supersedes the withdrawn v1, which recommended a syscall trap on the strength of a true premise (our migcom cannot match the frozen server) carried one step too far into a false conclusion (therefore MIG is unusable) — when the frozen server could simply be regenerated too. Detailed mach_port sub-plan: the mach_port MIG client. Background: MIG explained. Measured on gershwin-on-nextbsd; Apple sources from apple-oss-distributions/xnu and bootstrap_cmds-138.