← Back · sub-plan of the NextBSD graphics plan · sibling of the fallback graphics plan · decision record: arm64 KMS options
The target is the real driver, not a framebuffer stand-in: /dev/dri/card0 and a render node, dynamic resolution, and hardware-accelerated OpenGL through Mesa's virgl. This has never worked on any BSD. What follows is a complete specification of what stands between here and there — and the two findings that make it a substantially smaller project than it looked.
Finding 1 The LinuxKPI virtio shim can live entirely in our repo. No FreeBSD base change is required. This was the thing that appeared to make virtio-gpu depend on upstream cooperation. It does not.
Finding 2 The entire userland half already ships. FreeBSD's Mesa builds the virgl Gallium driver unconditionally, on every architecture, and libdrm already installs virtgpu_drm.h. Accelerated OpenGL needs no ports work — only the kernel driver.
Two deliverables that are often conflated. Both are in scope; the first is a prerequisite for the second.
| KMS | Accelerated OpenGL | |
|---|---|---|
| Device node | /dev/dri/card0 | + /dev/dri/renderD128 |
| Feature bits | VIRTIO_F_VERSION_1 | + VIRTIO_GPU_F_VIRGL, _CONTEXT_INIT |
| Ioctls | modeset, dumb buffers | + EXECBUFFER, GET_CAPS, RESOURCE_CREATE_3D |
| Userland | Xorg modesetting, any Wayland compositor | Mesa virgl Gallium driver |
| Host must offer | virtio-gpu-pci | virtio-gpu-gl-pci + GL display backend |
| Gives you | dynamic resolution, no tearing | GPU-accelerated GL |
Both are achievable with kernel work alone. Note the last row: the host has to offer 3D too — plain virtio-gpu-pci is 2D scanout however good the guest driver is.
More than expected, because the bochs and vboxvideo ports paid for most of the helper layer.
| Piece | Status |
|---|---|
DRM core on both arches (drm.ko, ttm.ko, dmabuf.ko) | ✅ builds, resolves, ships as kexts — arm64 proven in PR #33 |
Helper layer (drm_simple_kms_helper, drm_gem_vram_helper, drm_gem_atomic_helper, drm_format_helper, drm_gem_framebuffer_helper) | ✅ shipping as IOGraphicsExtras.kext |
| Out-of-tree LinuxKPI extension pattern | ✅ proven — graphics/include/linux/genalloc.h is exactly this |
| Kext packaging, autoload, symbol-resolution gate | ✅ ko2kext.sh, IOKit matcher, check-kext-symbols.py |
Boot-test harness that proves load → bind → card0 | ✅ used for bochs and vboxvideo |
| Mesa virgl + libdrm virtgpu | ✅ already in ports — see §3 |
| LinuxKPI virtio shim | ❌ does not exist anywhere |
drm_gem_shmem_helper | ❌ not in drm-kmod |
| The driver | ❌ never worked on any BSD |
graphics/mesa-dri/Makefile:
GALLIUM_DRIVERS= virgl zink
OPTIONS_GROUP_GALLIUM= crocus i915 iris llvmpipe panfrost r300 r600 radeonsi svga
OPTIONS_EXCLUDE+= ${ARCH:Naarch64:C/.+/panfrost/}
OPTIONS_EXCLUDE+= ${ARCH:Namd64:Ni386:Nx86_64:C/.+/crocus i915 iris svga/}virgl is in the unconditional assignment, not the options group — it is always built. And unlike panfrost (aarch64-only) or the x86 drivers, it carries no architecture exclusion, so it is built on amd64 and aarch64 alike. graphics/libdrm's pkg-plist installs include/libdrm/virtgpu_drm.h unconditionally.
So the moment a render node appears, Mesa can drive it. No ports work is in scope for this project.
Note: virglrenderer — the host-side library — is not in FreeBSD ports at all. That only matters for running FreeBSD as a virtualisation host, which is out of scope; our hosts are macOS running UTM/QEMU, which bundle their own.
FreeBSD already has a working virtio-gpu driver — sys/dev/virtio/gpu/virtio_gpu.c, 734 lines plus a 454-line header, by andrew@ (sponsored by Arm Ltd, 2023). It speaks the virtio-gpu protocol correctly today. So the obvious question is whether to grow that into a KMS driver rather than vendor Linux's.
Three genuinely distinct routes:
virtio_gpu(4) into a DRM driverKeep: the transport. It already does virtqueue_enqueue / virtqueue_notify / TRANSFER_TO_HOST_2D / RESOURCE_FLUSH against FreeBSD's native virtio API, correctly, with no shim.
Write: everything else — the DRM device, GEM objects, modesetting, the full ioctl surface, dma-buf, the render node, and 3D command submission.
The catch that sinks it: there is no FreeBSD DRM core other than drm-kmod. DRM on FreeBSD is drm-kmod. So “extending the base driver” still means linking against drmn.ko — you do not escape drm-kmod, you only give up Linux's driver. And it puts the work in FreeBSD base, where D32372 has waited five years and D55012 six months.
~4,000 lines vendored from v6.12, plus the ~600-line LinuxKPI virtio shim, plus shmem GEM. Everything lives in our repo, in the pattern already proven twice.
The hybrid: write the driver in BSD style against drmn.ko, but talk to the device through FreeBSD's native virtio API. This needs no LinuxKPI virtio shim at all — which is a real attraction, since the shim is the least conventional part of route B.
But you then hand-implement the GEM layer, the modeset paths, and — critically — the entire virtgpu uapi.
This is the argument Mesa's virgl Gallium driver is already installed and already built for both architectures (§3). It talks one specific kernel ABI: the ioctls in drm/virtgpu_drm.h — EXECBUFFER, GET_CAPS, RESOURCE_CREATE, RESOURCE_INFO, MAP, WAIT, CONTEXT_INIT, TRANSFER_TO/FROM_HOST — with exact structure layouts, exact semantics, and exact error codes.
That ABI is the contract, and it is not ours to design. Vendoring Linux's driver satisfies it by construction, because it is the implementation the ABI was defined by. Routes A and C require reimplementing it faithfully enough that an unmodified Mesa works — and every divergence surfaces as a rendering corruption or a hang inside a userland stack we did not write and cannot easily instrument.
Weighed against each other:
| A — extend base | B — vendor on drm-kmod | C — native on drm-kmod | |
|---|---|---|---|
| Mesa uapi fidelity | hand-written, must match exactly | exact by construction | hand-written, must match exactly |
| Needs LinuxKPI virtio shim | no | yes (~600 lines, no base change) | no |
| Needs shmem GEM | yes (or a BSD equivalent) | yes | yes (or a BSD equivalent) |
| Still depends on drm-kmod | yes — unavoidable | yes | yes |
| New code we own forever | ~4,000+ lines, no upstream | ~600 (shim) + deltas | ~4,000+ lines, no upstream |
| Upstream fixes flow to us | no | yes | no |
| 3D / virgl | write from scratch | included | write from scratch |
| Where the work lands | FreeBSD base (slow review) | our repo | our repo |
Route B. The shim is the price of admission and it is bounded, well-understood, and now known not to require a base change. Routes A and C trade ~600 lines of shim for ~4,000 lines of novel driver plus the obligation to reimplement a uapi we do not control, against a userland we cannot change, with no upstream to inherit fixes from. Route A additionally puts the work in base, on a review path this project has twice watched stall for years.
Base's driver is still useful Not as a foundation, but as a reference. virtio_gpu.c:441-489 is a known-correct example of the exact virtqueue dance our shim must reproduce — feature negotiation, virtio_alloc_virtqueues, then enqueue → notify → poll under one lock. It is the best available evidence of how these calls are meant to be sequenced on FreeBSD, and it should be read closely during Phase 1.
The apparent blocker was that D32372's shim calls vq_ring_must_notify_host() and vq_ring_notify_host(), both static in sys/dev/virtio/virtqueue.c (confirmed on both branches). That is why it shipped with a companion base patch, D32371, to export them — and that patch has sat in Needs Review since 2021.
It is not needed, for two reasons:
virtqueue_notify() already is Linux's virtqueue_kick(). Read virtqueue.c: it does bus_dmamap_sync(PREWRITE) + mb() + if (vq_ring_must_notify_host()) vq_ring_notify_host() + reset vq_queued_cnt. The notification suppression that kick_prepare() performs is already inside it.virtqueue_kick(). It uses the split kick_prepare/notify form at exactly four sites (virtgpu_vq.c:434,438,478,484).So the mapping is simply:
bool linux_virtqueue_kick_prepare(struct virtqueue *vq)
{
virtqueue_notify(vq->bsdq); /* does the suppression check itself */
return (false); /* driver then skips its own notify() */
}This performs the MMIO under the driver's qlock — which is precisely the convention every native FreeBSD virtio driver already follows, including base's own virtio_gpu.c. Zero spurious notifications, no base change, nothing to wait for.
D32372 is a sketch, not a baseline It does not compile — three errors, including returning values from void functions. It is also stale twice: virtio_alloc_virtqueues() lost its flags argument in 180c02405b12 (2023-09-18), and v6.12 replaced the callbacks[]/names[] signature with struct virtqueue_info — there is no virtio_find_vqs_ctx in 6.12 at all. Read it for the namespacing idea; write the rest fresh.
FreeBSD and Linux both define struct virtqueue and virtio_device. D32372 handled this with #define virtqueue linux_virtqueue plus __asm__ symbol renaming. Keep the __asm__ half — it guarantees the emitted symbol is linux_virtqueue_* and can never collide in the kernel linker. Drop the object-like #define: it leaks into every subsequent header and rewrites any unrelated identifier named virtqueue, failing far from its cause.
Our build compiles vendored Linux sources and the shim as separate translation units with separate include sets, so we can simply name the shim struct struct virtqueue and never include <dev/virtio/virtqueue.h> in any TU that sees it. Only linux_virtio.c sees both namespaces, and there one #define bsd_virtqueue suffices.
graphics/include/linux/virtio.h pure additions — LinuxKPI has
graphics/include/linux/virtio_config.h ZERO virtio files on main or
graphics/include/linux/virtio_ring.h releng/15.1, so no conflict is
graphics/include/linux/virtio_ids.h possible. Same pattern as our
graphics/include/linux/virtio_gpu.h existing linux/genalloc.h.
graphics/include/uapi/linux/virtio_gpu.h verbatim from v6.12
graphics/linuxkpi_virtio/ the shim .ko
linux_virtio.c sglist flattening, vq alloc, config space
linuxkpi_virtio_freebsd.c MODULE_VERSION + MODULE_DEPEND
Makefile EXPORT_SYMS=YES
graphics/drivers/gpu/drm/virtio/*.c vendored v6.12 + #ifdef __FreeBSD__
graphics/drivers/gpu/drm/drm_gem_shmem_helper.c → into drm_extra_helpers
graphics/virtio_gpu_drm/ driver .ko + newbus attach glueSymbol resolution is depth-1 only kern_linker.c:940-943 looks up symbols in file->deps[] with deps = 0 — transitive dependencies are not searched. Every module must MODULE_DEPEND on everything it uses directly. This is why i915kms re-declares linuxkpi and dmabuf even though drmn already depends on them, and why our drm_extra_helpers pattern works. It is now documented rather than folklore.
MODULE_DEPEND(virtio_gpu_drm, drmn, 2, 2, 2);
MODULE_DEPEND(virtio_gpu_drm, linuxkpi, 1, 1, 1);
MODULE_DEPEND(virtio_gpu_drm, dmabuf, 1, 1, 1);
MODULE_DEPEND(virtio_gpu_drm, drm_extra_helpers, 1, 1, 1);
MODULE_DEPEND(virtio_gpu_drm, linuxkpi_virtio, 1, 1, 1);
MODULE_DEPEND(virtio_gpu_drm, virtio, 1, 1, 1);Naming Do not name the module virtio_gpu. FreeBSD base already has MODULE_VERSION(virtio_gpu, 1) in sys/dev/virtio/gpu/virtio_gpu.c. That would be a name clash on top of the device clash. Equally: never add device virtio_gpu to the NEXTBSD kernel config. It is optional, absent from every GENERIC and from sys/modules, so it is not in our kernel today — and it must stay that way. It is a newbus driver that would compete for the same device, and it registers at VD_PRIORITY_GENERIC+10 against efifb's +1, so merely compiling it in displaces the console driver that works.
Unexpected advantage shmem backing sidesteps nextbsd-kernel#71 entirely. That defect blocks any driver whose buffers live in device memory: once a VRAM BO is pinned for scanout, the fault path hands a device-BAR pfn to a function that assumes managed shmem pages, and the faulting thread never wakes. It currently blocks bochs and vboxvideo, and it will block vmwgfx.
virtio-gpu's buffers are shmem — system memory — so they take the same branch i915 does, which is verified working on real hardware. The shmem helper was already the largest single cost in this plan; it turns out to also be what keeps virtio-gpu clear of the defect that stops every VRAM-backed driver.
Caveat: blob resources / host-visible memory do map device memory and would re-enter that territory. They are already out of scope (FreeBSD's 12-method VIRTIO_BUS interface has no get_shm_region), but if that bus method is ever added, this constraint returns with it.
virtio-gpu is built on shmem-backed GEM objects — virtgpu_object.c:145 calls drm_gem_shmem_get_pages_sgt(), and that is where all guest-page DMA mapping happens. It is not optional or incidental, unlike in the fallback driver where a TTM substitution works.
drm-kmod does not ship it. Two routes:
Vendor Linux's drm_gem_shmem_helper.c | Clean-room BSD implementation | |
|---|---|---|
| Size | 782 lines, mechanical | Comparable, but original |
| Licence | GPL-2.0-only — breaks our or-later invariant | Ours to choose |
| Dependencies | In good shape: drm_gem_get_pages/put_pages are exported by drm-kmod's drm_gem.c; LinuxKPI has shmem_fs.h, dma_map_sgtable(), set_pages_array_wc() | Built on FreeBSD vm_object/OBJT_SWAP directly |
| Side benefit | Unblocks drm_fbdev_shmem.c, which sits in drm-kmod's tree building into nothing | Same, and upstreamable to base |
Shared prerequisite This helper is not only virtio-gpu's cost. Hyper-V's DRM driver is shmem-backed too (DRM_GEM_SHMEM_DRIVER_OPS, drm_fbdev_shmem_setup()), so whatever is built here unlocks hyperv_drm as a comparatively cheap follow-on — and FreeBSD already ships the Hyper-V transport in base (hv_vmbus), where LinuxKPI has no virtio bus at all. See the other-hypervisors section.
It goes further than that. A survey of upstream drivers found that almost every small driver migrated to shmem years ago, so this one helper gates six of them: virtio-gpu, hyperv, ast, mgag200, cirrus and simpledrm — including ast, the ASPEED BMC driver that real arm64 server boards actually need. The helper is the largest single cost in this plan, and it is not this plan's cost alone.
This is decision V2, and it is the licensing fork for the whole project. Note the reason upstream rejected it on PR #119 — conflict with migrating code into FreeBSD base — does not apply to an out-of-tree NextBSD kext. What does apply is our own graphics/README.md invariant, which a vendored copy would break.
Both are hours, and either could invalidate the work that follows.
kldload hang. dsl@ and evadot report drm-kmod 6.12 hanging early on arm64. If that is a drm.ko core path rather than amdgpu-specific, arm64 is blocked regardless of this driver. Testable now by loading our arm64 IOGraphics.kext in UTM.virtio-gpu-gl-pci with a GL-capable backend, and whether Apple's Virtualization.framework exposes virgl at all. This decides whether the 3D half is reachable on your hardware or only under plain QEMU.linuxkpi_virtio.koSix headers plus linux_virtio.c. Out-of-line functions to implement, each with its FreeBSD backing:
| Linux | FreeBSD | Trap |
|---|---|---|
virtqueue_add_sgs | virtqueue_enqueue() over a flattened struct sglist | must return exactly -ENOSPC — virtgpu_vq.c:467 tests it literally |
virtqueue_get_buf | virtqueue_dequeue() | not virtqueue_poll(), which spins forever |
virtqueue_kick_prepare / notify | virtqueue_notify() | see §4 |
virtqueue_enable_cb | virtqueue_enable_intr() | polarity is inverted — FreeBSD returns 1 when work is pending |
virtqueue_disable_cb | virtqueue_disable_intr() | |
virtio_reset_device | virtio_stop() | |
config->find_vqs / del_vqs | virtio_alloc_virtqueues() (3 args) / virtqueue_free() | v6.12 uses struct virtqueue_info |
config->get / set | virtio_read/write_device_config() | |
config->get_shm_region | no equivalent | return false; rules out blob resources |
Header-only work: virtio_find_vqs, virtio_has_feature (note FreeBSD's virtio_with_feature takes a mask, Linux's takes a bit index), virtio_cread_le/cwrite_le, virtio_device_ready, dev_to_virtio, module_virtio_driver (LinuxKPI has no module_driver()). Also add dma_sync_sgtable_for_device(), absent from LinuxKPI on both branches.
Per V2: vendor or clean-room. Add to drm_extra_helpers, which already carries the four other files drm-kmod omits.
Not PR #119's Linux 5.5 snapshot — that calls drm_fbdev_generic_setup(), which no longer exists. Vendor fresh, apply #ifdef __FreeBSD__ deltas in our established style.
Write from scratch; D32372 provides none, and this is exactly where the 2021 attempt died (page fault at 0x30 in drm_sysfs_minor_alloc, because virtgpu_bsdmodule.c passed an entirely uninitialised struct linux_virtio_device into virtio_gpu_probe()).
virtio-gpu is a virtio_driver, not a pci_driver, so LinuxKPI's linux_pci.c routing does not apply — no .name = "drmn" trick here. Use VIRTIO_DRIVER_MODULE + VIRTIO_SIMPLE_PNPINFO(..., VIRTIO_ID_GPU, ...), and return better than BUS_PROBE_DEFAULT. Base's own sys/dev/virtio/gpu/virtio_gpu.c is the working reference for the whole sequence.
card0, modeset, dynamic resolution, and the efifb → drmfb handover. Then negotiate VIRTIO_GPU_F_VIRGL and _CONTEXT_INIT, expose the render node, and confirm Mesa's virgl driver binds it.
Assert the handover ordering explicitly in the boot test. On arm64, ArmVirtQemu's VirtioGpuDxe allocates the firmware framebuffer as a virtio-gpu resource on the device. Our device reset destroys it, so drm_aperture_remove_conflicting_framebuffers() must run first. On amd64 with bochs, efifb pointed at a PCI BAR that survives a reset, so sloppy ordering still looked fine. Here it would not.
VirtIOGraphics.kext for amd64 and arm64, through the existing ko2kext.sh path, the symbol-resolution gate, and the boot test.
R1 — silent memory corruption The scatterlist→sglist flattening can lose the readable/writable boundary. sglist_append_phys() coalesces physically adjacent ranges (subr_sglist.c:89-90). If the last readable segment is adjacent to the first writable one, they merge — and the device gets write access to our command buffer. virtqueue_enqueue() also asserts readable + writable == sg->sg_nseg, so the counts must be post-flattening segment counts, not Linux sgs[] element counts.
Mitigation: fill sg_segs[] by hand over sglist_init() rather than using the append helpers, or explicitly detect and reject the merge. Both call sites hold a spinlock and pass GFP_ATOMIC, so pre-allocate per-vbuf sglists at virtio_gpu_alloc_vbufs() time. Note sgs[1] can be a chained multi-page list — walk sg_next(), do not assume sg_is_last() as D32372 did.
R2 — a field that is a function here vq->num_free is read as a struct field inside wait_event() predicates (virtgpu_vq.c:338,341,469). FreeBSD exposes it only as virtqueue_nfree(vq). The predicate is re-evaluated after each wakeup with nothing in between to refresh a cached value, so a stale field deadlocks. Mitigation: refresh it inside get_buf/add_sgs (the wakeups come from the dequeue worker after it drains), or patch the three sites under #ifdef __FreeBSD__ per our vendoring convention.
R3 — capability ceilings Three limits that cap what will work rather than breaking it outright. (a) No get_shm_region bus method exists, so blob resources / host-visible memory are out. (b) VIRTIO_MAX_INDIRECT is 256 descriptors — fine for 2D, potentially tight for large virgl command buffers. Pass vqai_maxindirsz = 256 and negotiate VIRTIO_RING_F_INDIRECT_DESC; D32372 passed 0, silently disabling indirect descriptors entirely. (c) lkpinew_pci_dev() does not call linux_pdev_dma_init(), so a hand-built struct device has dma_priv == NULL and the Linux DMA API will null-deref. Either parent the DRM device on a real pci_dev, or ensure virtio_has_dma_quirk() is true (the normal QEMU case) so the sg_phys() branch is taken instead.
Fully worked in §4. The short form: DRM on FreeBSD is drm-kmod, so no route escapes it; and Mesa's virgl driver — already installed, already built for both arches — talks a uapi we do not control. Vendoring satisfies that ABI by construction; hand-writing it means reimplementing it exactly against a userland we cannot change and cannot easily debug. The counter-argument worth taking seriously is that route C needs no LinuxKPI virtio shim at all, which is the least conventional part of the recommended route.
Resolved by §4 — no base change is required, and LinuxKPI has zero virtio files to conflict with. Worth recording that upstreaming to drm-kmod is not the alternative: drm-kmod deleted linuxkpi_gplv2.ko in 2023 and is actively shrinking its out-of-tree LinuxKPI surface, so a shim would not be welcome there.
drm_gem_shmem_helper.c, or clean-room it? blockingThe licensing fork. Vendoring is mechanical but is GPL-2.0-only, breaking the or-later invariant stated in our own graphics/README.md. A clean-room implementation on FreeBSD vm_object/OBJT_SWAP keeps the invariant, is upstreamable to base, and would also unblock drm_fbdev_shmem.c — at the cost of being original VM work rather than a port. Whichever is chosen, the README should be amended or reaffirmed deliberately.
The 3D path adds EXECBUFFER, capsets and context-init on top of a working modeset driver. Landing KMS first gives a testable milestone and a shippable intermediate; going straight for 3D avoids reworking the ioctl surface twice. Note Phase 0 may decide this for us — if the host cannot present virtio-gpu-gl, 3D is untestable regardless.
Not virtio_gpu — base owns that MODULE_VERSION. Proposed virtio_gpu_drm for the module and VirtIOGraphics.kext for the bundle. Avoid linuxkpi_virtio verbatim for the shim too: it is a plausible future base module name, and drm-kmod's linuxkpi_video collides with base for exactly that reason.
arm64 is the motivating case, but virtio-gpu is equally the right driver for amd64 VMs, where it would supersede bochs for anything using QEMU/KVM. Shipping both also means the amd64 CI lane can exercise the driver, which arm64 CI currently cannot.
Per R3(a), no get_shm_region bus method exists in FreeBSD's 12-method VIRTIO_BUS interface. Returning false degrades cleanly. Adding it would be a base change — the only place in this project where one is even arguably needed.
device virtio_gpu decidedRecorded as a standing constraint rather than a decision to revisit. See the naming callout in §5.
virtio-gpu-gl-pci with a GL backend, and whether Apple's Virtualization.framework exposes virgl at all. Phase 0 work — it decides whether the 3D half is reachable on the target hardware.kldload hang on drm-kmod 6.12 touches drm.ko core paths. Phase 0 work.drm_gem_shmem_helper.c builds against our stack — its LinuxKPI gap was not assessed.dma_buf completeness for virtgpu_prime.c's export/attach path. Consider compiling that out initially.Sub-plan of the NextBSD graphics plan. Sibling of the fallback graphics plan, which solves a different problem (unsupported hardware) and does not block or depend on this one. The arm64 KMS options page is the decision record showing how this route was chosen over five alternatives. Every factual claim was checked against a primary source; §10 lists what was not.