NextBSD graphics: the NVIDIA userland / display layer

Companion to the per-branch kext porting plan. The kexts autoload on real hardware and the KMS device is up — but the screen is black, because a GPU driver is two layers and we shipped only the kernel one. This plan settles every option for the second layer: display paths, the X-server ABI question, the userland file set, multi-version co-existence, whether the libs can live in the kext, and whether Wayland changes the game. Draft, July 2026 · grounded in five parallel research passes + on-hardware testing (RTX 3060 Ti, 0x2489).

0. Your questions, answered — jump table

1. The layering model — why the kexts alone give a black screen

A GPU driver is two layers. We built the first and skipped the second:

The kernel modules expose device nodes. Nothing in the kernel can draw an X screen — X always needs a userland display driver (DDX) to consume those nodes. With no NVIDIA DDX present, XLibre auto-picks the generic vesa driver, which claims the PCI GPU before the DRM path is even considered, and the monitor drops signal. That is the exact black screen observed on hardware.

2. The three display-path options

Option A — Driver "nvidia" (the NVIDIA X11 userland)  RECOMMENDED — VALIDATED ON HW

Load NVIDIA’s own X driver nvidia_drv.so. It talks to /dev/nvidia0 (+ NVKMS via /dev/nvidia-modeset) directly and never touches the DRM/modesetting/GBM path. This is FreeBSD’s supported way and the low-risk target. Needs the userland package (§4). Driver "nvidia" is not an alternative to the userland — nvidia_drv.so is the userland; it’s the reason we need it.

Option B — modesetting on nvidia-drm

B1 — generic / dumb-buffer form: DEAD (proven on HW). The stock X.Org modesetting DDX on /dev/dri/card0 with no NVIDIA userland present. Tested on hardware with an explicit Driver "modesetting" + BusID config:

(EE) open /dev/dri/card0: Operation not supported by device   ← nvidia-drm rejects generic KMS
(EE) Device(s) detected, but none match those in the config file.
Fatal server error: no screens found

And repeated startx panicked the kernel in the nvidia-drm KMS path. Unaccelerated modesetting allocates scanout via DRM_IOCTL_MODE_CREATE_DUMB (linear, CPU-mappable); NVIDIA scanout memory is tiled/compressed VRAM the driver manages itself, so the ioctl returns EOPNOTSUPP. With no NVIDIA GBM userland installed there is no other allocation path — hence the wall.

Option C — Wayland (EGL + GBM + KMS)  Phase 2 — dodges both X problems, but a FreeBSD userland gap

A Wayland compositor drives the GPU through nvidia-drm KMS (which we have) + libEGL_nvidia + a GBM backend — loading neither the ABI-versioned X DDX nor GLX. Architecturally it erases both of our X-side blockers, but the required userland GBM stack is Linux-centric (§7). Shares its entire userland set with Option B2.

Which kernel modules + which libraries each path uses

The kexts already carry all three kernel modules; the question is which userland each path pulls from the per-version bundle. There are really two userland sets plus a shared core — and all of it is bundlable per-branch (§5, §7):

Key insight: B2 and C share the same EGL/GBM userland — packaging that one set is dual-purpose. Path A is the outlier (DDX + server-side GLX) and the only path that ignores nvidia-drm entirely. libnvidia-glcore is common to all. No userland lib lives in the kext; they live in the sibling per-branch bundle (§5) — the kext stays purely kernel.

3. The X-server question: XLibre vs classic Xorg

The ABI numbers (verified on the box + upstream): mainline released xorg-server-21.1.x = video ABI 25.2 (stable across the whole 21.1 series); mainline dev master is already 27.0; XLibre forked from master and added +1 → 28.0 (the 28.1 we see is XLibre’s CONFIG_LEGACY_NVIDIA_PADDING ScreenRec-padding variant, added specifically to keep the NVIDIA binary working). NVIDIA 595’s nvidia_drv.so targets ABI 25.

Does it matter which we test on? Yes for isolating variables, no for the packaging. Both host the same 595 userland with the same files — only the host differs. Use classic xorg-server-21.1.24 as the clean control (removes the “is it XLibre’s shim misbehaving?” variable); confirm on XLibre-25 for the real target.

How important is the XLibre ABI / do we bump every release? Within the entire 25.x era the video ABI is pinned at 28 — 25.0, 25.1, 25.2 all hold it; it did not track XLibre’s ~weekly release cadence. The dynamic is inverted from what we feared: XLibre absorbs the compatibility burden for the closed binary and re-shims after each ABI break, rather than expecting NVIDIA to rebuild. The residual risks are (a) the next major XLibre bump, at which point XLibre — not us — must re-shim, and (b) the documented flakiness. Neither forces a per-year action on our side. Net: we are not chasing a moving target; classic Xorg remains the guaranteed host if XLibre’s best-effort support regresses.

4. The userland package — manifest, minimal set, per-branch ABI cutoffs

Can we pkg install FreeBSD’s nvidia-driver as-is? No. It bundles the kmod we’ve replaced with kexts, targets FreeBSD paths, and version-locks differently. We build our own package from the same NVIDIA-FreeBSD-x86_64-595.84.tar.xz we already fetch for the kmod build — extract just the userland .so set and stage it, exactly as we did for the kexts. (FreeBSD models the X server choice with port FLAVORS xorg / xlibre; the nvidia_drv.so binary is the same NVIDIA-precompiled ABI-25 blob either way.)

Minimum file set for just a display (then additive)

Validate the exact runtime closure with ldd nvidia_drv.so and ldd libglxserver_nvidia.so against the real 595.84 binaries before trimming; confirm the tarball’s .manifest (file→dest map) by unpacking it. Sources: NVIDIA 595.84 FreeBSD README “Installed Components”; FreeBSD x11/nvidia-driver master Makefile + pkg-plist.

Per-branch X-server ABI cutoffs (which branches are even hostable)

5. Co-existence — versioned files, the activator, and the “bake it into the kext” idea

Which files are unique vs which collide

You were right that the kext paths are unique so the kernel side has no collision. The userland is a split picture:

So dumping two branches into shared dirs collides on the entry points. The fix is the same “unique directory” trick as the kexts: give each branch its own tree, then activate one.

Why GLVND doesn’t make versions co-exist

GLVND is vendor-neutral, not version-neutral. It multiplexes Mesa ↔ NVIDIA beautifully (per-X-screen vendor selection), so an Intel/AMD-Mesa GPU and NVIDIA co-exist with zero conflict. But every NVIDIA branch names its vendor "nvidia" and ships the single fixed soname libGLX_nvidia.so.0 (and libEGL_nvidia.so.0) — there is no nvidia-580 vs nvidia-470 namespace. So one NVIDIA version is active at a time, which is fine because one GPU branch is resident per boot anyway (the kernel-namespace wall from the kext plan). This holds identically on the EGL/Wayland path.

The activator (Debian’s model, adapted)

Debian runs exactly this: glx-diversions + update-glx divert the canonical files and flip the whole set atomically. Our adaptation, keyed to the autoloaded branch / PCI-ID (not /dev/nvidia, which can’t tell you the branch):

• X DDX — no symlink needed: point the branch’s OutputClass/Device ModulePath at its own dir. X loads it directly.
• GL client libs + GLVND JSON — resolved globally by ld.so/GLVND, so symlink the active branch’s entry points into the canonical location (respect the no-ldconfig-boot-job doctrine: reach the libs through an already-cached canonical dir / the sanctioned STANDARD_LIBRARY_PATH lever, not a per-branch ld.so.conf.d rewrite).
• Atomicity — a half-flipped set is an instant black screen; stage into a shadow dir and flip via one rename() of an active → <branch> directory symlink.
• Where it runs — as an ISO/first-boot installer step (detect GPU → lay down the one active branch), not a runtime hot-switch. A reboot-to-reactivate is enough; there is no user demand for live NVIDIA-version switching.

“Can we bake the libs into the kexts? Is it just another resource like firmware?”

Storage-wise, yes — a .kext is a bundle and can carry arbitrary Resources/, so libs inside NVIDIAGraphics595.kext would get a unique, versioned path for free. But firmware and userland libs are consumed by different layers, and that’s the catch:

So the firmware analogy holds for storage but not for resolution. In practice: the DDX loads cleanly from the bundle (X’s ModulePath can point into …/NVIDIAGraphics595.kext/Contents/Resources/xorg), but the GL client libs still need the thin activation above to appear on the linker/GLVND search paths.

Does Apple do this? Not libs-inside-a-kext. Apple co-locates userland GPU driver bundles as siblings to kexts in /System/Library/Extensions/ — the Metal drivers (AMDMTLBronzeDriver.bundle, AppleIntelGraphicsMTLDriver.bundle), and DriverKit .dext drivers are userland bundles too. So the Apple-idiomatic shape is:

/System/Library/Extensions/
    NVIDIAGraphics595.kext      ← kernel (nvidia-drm chain)   [shipped]
    NVIDIA595.bundle/           ← userland X/GL (nvidia_drv.so + libs)  [new]
        Contents/Resources/xorg/nvidia_drv.so
        Contents/Resources/lib/libGLX_nvidia.so.595.84  …

— a per-branch sibling bundle, versioned in lockstep with its kext, giving the exact unique-path co-existence you want while keeping kernel and userland as one installable unit. Both variants (libs-in-kext vs sibling-bundle) are viable; the sibling bundle is closer to Apple and keeps the kext purely kernel.

6. Is libGLX_nvidia.so.0 strictly required? (Linux included)

No. It is the GLVND client vendor library for GLX (OpenGL over X11) — nothing more. What runs without its GLX code:

So an X desktop comes up with nvidia_drv.so and no libGLX_nvidia — you just have no hardware GLX (GLX apps fail, or fall to Mesa software GLX if present). The one gotcha: because NVIDIA points its Vulkan ICD at the same file, the file must exist for Vulkan even though no GLX runs. And there are hardware-GL-on-X11 bypasses that route around NVIDIA GLX entirely — Zink (OpenGL→Vulkan on NVIDIA’s Vulkan ICD) and native EGL — useful to know, but not needed for our first goal.

7. Does Wayland change things?

How Wayland dodges the dumb-buffer wall

Mesa’s libgbm, on gbm_create_device(fd), reads the DRM driver name and dlopens lib<driver>_gbm.so — here nvidia-drm_gbm.so, a symlink to libnvidia-allocator.so — which produces properly tiled scanout buffers and a valid KMS framebuffer, succeeding exactly where the generic linear allocator returned EOPNOTSUPP. GBM has been NVIDIA’s default since driver 495 (EGLStreams is legacy); GNOME/KDE/wlroots all support NVIDIA via GBM; explicit-sync (the last big blocker) landed in 555.

What a Wayland session needs (and doesn’t)

• Needs: libEGL_nvidia.so.0 + GLVND EGL loader + 10_nvidia.json; libnvidia-egl-gbm.so.1 + 15_nvidia_gbm.json; libnvidia-allocator.so (→ nvidia-drm_gbm.so); libnvidia-glcore; a GBM-backend-aware Mesa libgbm; and a compositor that uses GBM. Kernel side already satisfied (modeset=1, card0).
• Doesn’t need: the ABI-versioned nvidia_drv.so DDX or any Xorg server-ABI module.
• XWayland (for X11/GNUstep apps) loads no xf86-video-* module either — again no DDX ABI problem — and accelerates via glamor-on-EGL/GBM; client apps still use client-side libGLX_nvidia.so.0, which is ABI-independent of the server.
• Multi-version: unchanged — libEGL_nvidia.so.0 is a single fixed soname too, so still one NVIDIA version per boot.

Packaging the EGL/GBM userland per-branch — and why it’s cleaner than X

Every file the EGL/GBM path needs (shared by Wayland and the revived modesetting+glamor path B2) is either version-unique (bundle it) or a fixed-name entry point with a clean environment-variable selector — so one per-branch NVIDIA<ver>.bundle carries the whole set and, for a compositor, nothing touches the canonical filesystem:

Why cleaner than the X path. On classic X, the GL client libs are loaded by arbitrary apps all over the system, forcing you to wire the active branch into global paths (the activator/symlinks of §5). But a Wayland compositor is one process that we launch — so a launch wrapper points every selector at the active bundle and touches nothing canonical:

exec env \
  GBM_BACKENDS_PATH=/System/Library/Extensions/NVIDIA595.bundle/Contents/Resources/lib/gbm \
  __EGL_VENDOR_LIBRARY_DIRS=…/Resources/share/glvnd/egl_vendor.d \
  __EGL_EXTERNAL_PLATFORM_CONFIG_DIRS=…/Resources/share/egl/egl_external_platform.d \
  LD_LIBRARY_PATH=…/Resources/lib \
  wayfire

The same env-selector wrapper works for path B2 (launch X with those vars set); path A’s DDX selects via ModulePath, and only path A’s GL client libs still need a global path because random X apps load them. Net: the JSONs and /usr/local/lib/gbm are not blockers — they’re fixed default locations overridden per-launch, so every branch’s Wayland userland co-exists on disk and the compositor points at whichever branch autoloaded.

8. Recommendation & phased roadmap

1. Phase 1 — ship the Driver "nvidia" X11 userland (595). Package the minimal DDX set (§4) from the 595.84 tarball as a per-branch sibling bundle NVIDIA595.bundle (§5); add the real Driver "nvidia" OutputClass (FreeBSD’s, with ModulePath into the bundle). Validate on classic xorg-server-21.1.24 first (clean ABI-25 control), then on XLibre-25 (the shipping target, auto-detect). Expand to full GL/EGL/Vulkan once a display is confirmed.
2. Phase 1b — the activator + per-branch bundles for 580/470/390. Detect GPU → activate one branch atomically at install/first boot. (340/304 = KMS-console only; document the limit.)
3. Phase 2 — the EGL/GBM userland (unlocks B2 and Wayland at once). Package the GBM closure (libEGL_nvidia, libnvidia-egl-gbm, libnvidia-allocator→nvidia-drm_gbm.so, libnvidia-glcore, the two JSONs) + a GBM-aware libgbm. Because this one set is shared by modesetting+glamor (B2) and every Wayland compositor (C), a single experiment settles both. Cheap first probe (no reboot): confirm the FreeBSD nvidia-driver-595.84 package actually ships a working nvidia-drm_gbm.so — the exact Linux-centric piece Agent-research flagged as unverified on FreeBSD. If present: try modesetting+glamor under X first (smaller change than standing up a compositor), then wayfire+seatd for native Wayland. Gated on FreeBSD GBM maturity, but the packages are already in the repo.

9. Open questions to verify on hardware

• ldd the real 595.84 nvidia_drv.so + libglxserver_nvidia.so to pin the exact minimal runtime closure; confirm the tarball .manifest format.
• RESOLVED ✓ Does XLibre-25.1.8’s auto-detect actually bring up Driver "nvidia" on this box? Yes — validated on hardware with the xlibre-nvidia-driver-595.84 userland, no xorg.conf needed. Classic xorg-server-21.1.24 remains the untested fallback if XLibre ever regresses.
• Confirm ModulePath-into-a-bundle is honored by XLibre’s loader (vs. requiring the DDX under the standard modules dir).
• Whether NextBSD/Gershwin wants classic Xorg as the default host on NVIDIA machines, or accepts XLibre’s best-effort NVIDIA support.
• Phase 2 only: does the FreeBSD 2026 nvidia-driver actually install a working nvidia-drm_gbm.so / libnvidia-egl-gbm, or must we port it?

10. The packaging recipe — building NVIDIA<NNN>.bundle in CI (repeatable per branch)

The userland ships in the same NVIDIA-FreeBSD-x86_64-<ver>.tar.xz the nvidia-build job already downloads for the kmods — as precompiled blobs, no compile step. Verified tarball layout (595.84):

Where it plugs in (two small changes, one auto-flow)

1. nextbsd-kernel-modules — a new tools/nvidia-mkbundle.sh <WRKSRC> <version> <outdir> (mirrors nvidia-portpatch.sh): stages the table above from $WRKSRC/obj into NVIDIA<NNN>.bundle and drops it into the same nvidia-kexts/ dir the wrap step already fills. One extra step in the nvidia-build job, right after “Wrap the chained NVIDIAGraphics595 kexts.”
2. Auto-flow through publish — no change needed: the publish job already does tar -C "$NDIR" -czf out/nvidia-kexts.tar.gz . over the whole dir, so the bundle rides into nvidia-kexts.tar.gz beside the kexts for free.
3. nextbsd-pkg — confirm/extend scripts/build.sh to extract the whole nvidia-kexts.tar.gz into stage/kexts/System/Library/Extensions/ (it already globs the kext tarballs; the bundle lands beside the kexts → ships inside NextBSD-kernel-extensions).

Why it’s repeatable for every future branch

nvidia-mkbundle.sh is version-parameterized, exactly like the kext wrap. Adding 580/470/390 is the same recipe against that branch’s tarball → NVIDIA580.bundle / NVIDIA470.bundle / NVIDIA390.bundle, each version-locked to its NVIDIAGraphics<NNN>.kext and co-existing on disk (§5). The per-branch install-time activator then points X’s ModulePath + the ld/EGL selectors at whichever branch autoloaded. So the “kernel kext + sibling userland bundle, both built from one tarball in one job, shipped in one package” pattern established here for 595 is the template every later branch reuses unchanged.