NextBSD: Linux support — the unified design

The three detailed plans already exist and are the source of truth for their layer; this document does not re-derive them. It adds the connective tissue, the real-box findings, and the decisions. Read top to bottom for the end-to-end story, or jump in:

1. The three layers
2. The rootfs — one root, Devuan, and the /compat layout
3. The kernel — and the PT_INTERP→SIGABRT defect
4. User identity (NSS) — so any user can run in it
5. The AppImage runtime
6. End to end: a user runs an AppImage
7. Phased plan & ticket map
8. Open decisions

1. The three layers

Everything stacks in three independent layers. A layer below is a prerequisite for the one above; a layer is inert (harmless, unmounted/absent) if the one below is missing.

2. The rootfs — one root, Devuan, and the /compat layout

2a. The base: Devuan

The debootstrap plan already made this choice; it holds. Devuan Excalibur is the default because it has the same Mesa 25.0.7 / libdrm 2.4.124 (built against 6.12) as Debian 13 trixie — an exact match for NextBSD’s drm-kmod 6.12-lts — but no systemd at all, and systemd’s postinst is the one thing that has actually failed under the Linuxulator.

2b. The /compat layout — one root, and why

The constraint is in the kernel, not in our layout. In sys/compat/linux/linux_util.c: linux_emul_path is one global string (:71, tunable as compat.linux.emul_path, CTLFLAG_RWTUN). On every Linux-ABI exec, linux_pwd_onexec() (:77) looks up that one global and calls pwd_altroot() to set the thread’s alt-root to it (:91; if the path is missing it falls back to the native root, :87). The alt-root is per-thread, but its value is that single global — there is no per-binary or per-process root selector. Hence one active Linux root: /compat/linux and /compat/devuan cannot both be live. Two simultaneous roots would require a kernel change to linux_pwd_onexec (a per-credential dispatch).

That one root serves three axes; only two of them force a choice of base:

2c. /lib64 — required, minimal, and why

Every unpatched x86-64 glibc binary carries PT_INTERP = /lib64/ld-linux-x86-64.so.2. The Linuxulator resolves that path under the alt root (a kernel lookup, not ld.so.conf), so the root must contain /lib64. Verified on the box: remove /compat/linux/lib64 and a bare chroot /compat/linux /bin/ls dies with SIGABRT (rc 134); restore it and rc=0. Minimal layout:

/compat/linux/lib64            -> usr/lib64          (symlink)
/compat/linux/usr/lib64/ld-linux-x86-64.so.2         (real file)
/compat/linux/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2   (real file, multiarch)

2d. The six ABI mounts (always present)

From the ABI-mounts plan, made by linux_abi_mounts() inside launchctl bootstrap at every boot (nextbsd-userland #244, fixing #190), even when no userland is installed:

$emul = compat.linux.emul_path. These are the compat instances; a path that is missing under $emul falls through to the native root (one extra negative lookup), which is how a self-contained AppImage runs with no userland.

3. The kernel — and the PT_INTERP→SIGABRT defect

The necessary kernel work is in the Linuxulator plan: bake linux_common + linux64 (sets kern.features.linux64), and build NVIDIA’s kext with the Linux-ioctl compat off. That is the whole of L1. What this section adds is a defect found on the real box that the plans did not anticipate.

4. User identity (NSS) — so any user can run in it

This is the layer that makes it any user, not just the one who built the root. Two distinct problems live here; conflating them is what muddied the earlier discussion.

4a. Resolution: “I am uid 5000” → “I am admin”

A host user (say admin, uid 5000, home /Local/Users/admin) execs a Linux ELF. It runs as uid 5000, and glibc calls getpwuid(5000) → NSS files → the root’s /etc/passwd. That file is the 20-line Devuan/Debian base — 5000 is not in it → NULL. The app cannot find its username or $HOME and misbehaves or fails.

The host resolves users with NextBSD’s own nss_directory_services module (/usr/lib/nss_directory_services.so.1, source in nextbsd-userland/src/nss_directory_services/; the host nsswitch is passwd: directory_services files). It is a FreeBSD nsdispatch(3) NSS source resolving from the DirectoryServices store.

4b. The options

4c. Provisioning: creating the accounts the base needs (the messagebus gap)

Distinct from resolution: some Linux packages need a system account to exist. The canonical one is messagebus. The dbus-system-bus-common postinst provisions it with a branch that keys on the base:

if command -v systemd-sysusers >/dev/null; then
    systemd-sysusers dbus.conf          # Debian: has systemd
else
    adduser --system --quiet --group messagebus   # Devuan: no systemd
fi

• On Devuan (no systemd-sysusers) → the adduser arm runs at install time → messagebus is created in place → no gap. This is one more reason Devuan is the right base: it removes the failure by construction.
• On Debian (has systemd-sysusers, which only runs at boot under a live systemd — never in a chroot) → the account wasn’t present when dbus’s core postinst ran its dpkg-statoverride --add root messagebus … under set -e → half-configured cascade (dbus, dbus-user-session, dconf-*, libgtk-3-0). Fix (reference, for a Debian root):

  groupadd -r messagebus
  useradd -r -g messagebus -d /var/run/dbus -s /usr/sbin/nologin messagebus
  dpkg --configure -a

Verified on the box: after the three lines, dpkg --audit is clean and all five packages are ii; systemd is in the tree only because dbus-user-session Depends: … libpam-systemd, systemd, and it is inert (no PID-1) in a root. Devuan sidesteps all of it.

5. The AppImage runtime

AppImages are self-contained by design — the payload bundles its own runtime. So an AppImage needs only the compat mounts (under the single emul_path, §2d) + /dev/fuse (FUSE) or just the ld-fix (extracted, §3) + identity (§4). It does not need a curated base — it rides whichever root the mounts are under, coexisting with the rl9 ports (§2b). The “curated client-lib set” in §5b is therefore for portable (system-lib) software and the rare non-self-contained AppImage — not for ordinary self-contained ones.

5a. Delivery: FUSE vs extracted

5b. What a portable GUI program needs from the root

(A self-contained AppImage needs none of this — it brings its own libs. This list is for portable programs, and the rare non-self-contained AppImage that links system libraries.)

• Curated client-lib set (nextbsd/nextbsd #526): the files NSS base accounts + the libraries the apps link (libGL/Mesa+/dev/dri, X11/XCB, fontconfig/freetype, and the app-specific SONAMEs). Defined against the installed base’s index; where the base ships a newer SONAME, pull the older one’s .so from a matching older codename’s .deb (apt-get download + dpkg -x) — the #526 recipe. (86Box specifically wants libfluidsynth.so.2 + librtmidi.so.5, both older than what trixie-class bases ship.)
• D-Bus — the session bus, the root’s own. GUI apps (GTK/Qt) use the D-Bus session bus (gsettings, portals), not the system bus. Two facts settle it: (1) a native FreeBSD dbus-daemon and a Linux ELF are different ABI/namespace — don’t bridge them; (2) these two apps don’t even need a running bus to launch (GTK/Qt degrade gracefully). If you want full gsettings, run the app under the root’s own Linux bus: dbus-run-session -- ./app. The native NextBSD dbus stays separate, for native consumers.
• GPU: NVIDIA’s Linux-ioctl compat is off, so a Linux app on an NVIDIA card gets llvmpipe; on Intel/AMD, Mesa/DRI drives NextBSD’s drm-kmod 6.12-lts through /dev/dri (provided by the root’s devfs) — that path works.
• Display: the app connects to the NextBSD display (X/Wayland) over the socket; the window appears on the desktop. (The GUI-server bridge is a separate track; a headless run only needs the above.)

6. End to end: a user runs an AppImage

1. admin (uid 5000) double-clicks 86Box.AppImage → the .app wrapper calls nextbsd-linux run ./86Box.AppImage.
2. The kernel Linuxulator (L1) translates; linux_pwd_onexec() sets the alt root to emul_path = the Devuan root (L2).
3. The six ABI mounts are already up (L2) — /proc, /sys, /dev/dri, /dev/shm, /dev/fd, /tmp.
4. Identity (L3, §4a v1): the wrapper has ensured 5000 admin … is in the root’s passwd/group (and mounted /Local/Users/admin if the app wants its real home). getpwuid(5000) → admin.
5. Runtime (L3, §5): curated libs load from the root; the relative-interp ld is a real file (so no SIGABRT); dbus-run-session gives a session bus; Mesa drives /dev/dri.
6. The window appears on the NextBSD desktop.

7. Phased plan & ticket map

8. Open decisions