16 — Native Plugin GUIs

Status · reconciled 2026-08-30. macOS embedding and resident edit graphs are implemented. Linux/X11 host support builds and has been exercised under Xvfb; a real x86_64 Linux CLAP editor plus touch input remains an M6 target test.

The macro layer (13 §4, 15) surfaces up to eight performance controls per slot — enough to play a patch, not enough to design one. Deep sound design needs the plugin’s own editor (Surge’s full panel, Dexed’s operators). This document fixes how Auvra embeds a CLAP plugin’s native GUI on the instrument’s touch surface — the “full-control escape hatch,” complementary to the macros, never a replacement.

It refines 07 §3 (the kiosk display server) and 14 §1 (the appliance render path): hosting native plugin GUIs forces a windowing-system decision, resolved in §3.

1. Where it fits

• Reached from the MIXER (15 §1). Each slot’s strip gets an Edit affordance that opens the plugin’s editor; it takes over the display behind a thin Auvra chrome bar (◀ back + plugin name). MIXER is build & set-up, and editing a plugin’s full parameter set is exactly that.
• Not on PERFORM. PERFORM is the glanceable, touch-first control surface driven by the macros; a dense mouse-era plugin panel is the opposite of that. The native GUI is for setup and deep edits between numbers, not the live gig.
• Complements macros, doesn’t replace them. You dial a sound in the native editor, then bind the handful of controls you want live to the slot’s macros (15 §2). Macros are the durable performance surface; the native GUI is the workbench.

2. What CLAP gives us

The clap.gui extension¹ is a negotiation between host and plugin. clack-extensions 0.1 implements the whole host side (PluginGui + HostGui), so this is glue, not new FFI.

Step	Call	Notes
Pick API	is_api_supported / get_preferred_api	We offer the native API for the display server: cocoa (Mac dev), x11 (appliance — §3).
Embedded vs floating	create(is_floating=false)	We always embed. A kiosk has no window manager, so a floating top-level window is meaningless — the editor lives inside our window.
Scale	set_scale	Feed the panel’s DPI so vector editors render crisply.
Size	get_size / can_resize / set_size	Fixed-size plugins dictate their size; resizable ones are fit to the editor region.
Parent	set_parent(Window)	Window::from_cocoa_nsview(ptr) / from_x11_handle(xid) — the plugin embeds into a child surface we own.
Show	show / hide	Map/unmap without destroying (fast re-open).
Teardown	destroy	Frees the plugin’s GUI resources.

Two things the host must provide for a plugin editor to actually run, beyond the calls above:

• clap.gui host callbacks (HostGui) — the plugin calls back to request_resize (it wants a new size), request_show / request_hide, and closed (a floating window was closed). We register these.
• clap.timer_support (HostTimerSupport) — most plugin editors drive their own repaint/animation from a host timer. Without it, the GUI opens frozen. The host registers the plugin’s timer and calls on_timer on the main thread; we tick registered timers once per frame. (On Linux, some editors also want clap.posix_fd_support to pump their event loop — added with the X11 path in §6.)

3. The platform reality

A plugin’s editor is drawn by the plugin, into a native OS window we hand it. That requires a windowing system to exist. On the appliance (07):

Path	Native GUI?	Why
X11 kiosk (bare Xorg, no WM / tiny WM)	✅	Proven — every Linux DAW embeds plugin editors this way.
Wayland — embedded	❌	Wayland has no cross-client embedding; clack itself notes “only Wayland does not support window embedding” (clack-extensions gui.rs). Plugin frameworks don’t implement it.
Wayland compositor + XWayland	⚠️	The plugin still speaks X11 via XWayland — a compositor on top of an X dependency. More moving parts, no benefit; reduces to X11.
Xvfb (hidden X) + framebuffer scrape	⚠️	Keeps a bare-KMS look, but is still X11, hidden; loses GPU accel for the plugin and adds capture + input round-trip latency. Fragile; wrong for a shipping instrument.
Bare KMS/DRM (the 14 §1 lean)	❌	We own the raw framebuffer — no windows to embed into. Categorical.
Plugin draws into our egui/GL texture	❌	CLAP has no “render into host surface” API. The plugin owns its window + toolkit. No escape hatch.

Every viable route on Linux routes through X11. The only real variable is whether X11 is visible and owns the panel (a real X kiosk — the sane choice) or hidden and scraped (Xvfb — the bad choice). There is no X11-free way to host a real plugin editor on Linux today; that’s a Wayland-maturity + plugin-ecosystem fact, not our limitation, and worth revisiting only if Wayland embedding matures.

Decision. The appliance runs a minimal X11 kiosk (bare Xorg, no window manager, or a ~200 KB one), and the whole display path — Auvra’s own egui UI included — renders through SDL3’s x11 backend, not bare KMS/DRM. Embedding requires the host window and the plugin window to share one display server, so enabling native plugin GUIs drags our UI onto X11 too. This is not “X beside KMS”; it is “our UI now runs under X.” X11 ≠ a desktop — a bare X server is a standard appliance pattern with a modest image footprint.

The macOS dev host is unaffected: it embeds via cocoa. All of §4–§5 is provable on the Mac before any of the X11 work lands.

4. The threading problem

This is the hard part, and it is threading, not pixels.

• clack_host::PluginInstance is !Send — it never leaves the thread that built it (12, auvra-clap).
• CLAP gui.* is [main-thread], and on macOS AppKit view mutations must run on the OS main thread.
• Auvra’s engine builds and owns plugin instances on a dedicated CLAP host thread (so patch switches never hitch the UI), and marshals main-thread calls (save_state, remote_controls) to it. That host thread is not the OS main thread — and because the instance is !Send, we cannot move it to the OS main thread to give it a GUI.

So the instance we want to edit was built on the wrong thread, and it can’t be moved. The resolution:

Decision — promote-on-open. The async engine is left untouched. Opening a slot’s editor builds a fresh instance of that plugin on the OS main thread, from the slot’s live (bundle, plugin-id, state-blob) — the state is captured from the running instance, so the editor opens on exactly the current sound — then hands that instance’s Send processor to the engine as the slot’s new audio source (retiring the old one through the existing disposal path, 12 §5). Now the on-screen editor and the sounding instance are the same object, living on the OS main thread. All the GUI/main-thread complexity is localized to the deliberate “open editor” action; the common path (play, switch, preload) never touches it.

flowchart LR
    subgraph MAIN["OS main thread — egui + SDL3"]
      UI["UI / event loop"]
      ED["Editor instance<br/>(!Send, main-thread)<br/>+ embedded plugin GUI"]
    end
    subgraph HOST["CLAP host thread"]
      OWN["Slot owners<br/>build / preload / state<br/>(!Send instances)"]
    end
    subgraph AUD["Audio thread"]
      ENG["Engine + mixer<br/>(Send processors)"]
    end
    UI -->|"open: (bundle, id, live state)"| ED
    OWN -->|"build async (unchanged)"| ENG
    ED -->|"swap in as slot source"| ENG

Costs, stated honestly. Opening an editor triggers one state-save + main-thread rebuild + audio-source swap — a brief, bounded hitch (covered by a spinner) and a momentary audio discontinuity (voices reset on the fresh instance). This is acceptable because it is a deliberate, occasional action, not a per-frame or per-switch cost. Why promote-on-open over simply building everything on the main thread: an instrument’s switches must stay silky, which the async host thread guarantees; trading a one-time glitch when opening a workbench for glitch-free performance switching is the right call.

Phase-2 refinements (§7): the audio swap is already click-free (the SwitchGraph release tail). Re-opening is made instant by keeping the edit graph resident — after the editor is closed its instances stay live (demoted to the host thread only on a patch switch), so re-opening any slot’s editor is a plain embed, no rebuild. (Done, Mac.)

5. The editor UX

• Fullscreen takeover. The editor fills the display behind a thin Auvra chrome bar: ◀ back, the plugin name, and (later) scale/size affordances. No floating windows — a kiosk has no window manager, and one editor at a time keeps the model simple and the main thread calm.
• Compositing. We embed straight into the SDL window’s content view and size the window to editor height + chrome. The plugin’s view sits at the parent’s origin (bottom-left in AppKit), leaving the top strip uncovered, where egui draws the chrome bar through the GL layer. This needs no intermediate NSView (no objc) — but it assumes the plugin origins its view at the bottom-left; a plugin that fills or centres its view instead would cover the chrome, and the fallback is a positioned intermediate child view (objc) as originally sketched.
• Modal. Editor mode suppresses the MIXER header, BROWSE overlay, and setlist nav, and discards any interaction that leaks through — so no stray patch switch, capture, or plugin swap can race the (UI-thread-owned) editor session. Opening BROWSE closes it; the brief open-capture round-trip shows a quiet “Opening editor…” placeholder, not the live MIXER. On OS-window close, the editor’s GUI is torn down (gui.destroy) on the final frame while the window is still alive, before the owner drops — the set_parent lifetime contract (§2) requires it.
• Touch honesty. Plugin editors are built for a mouse — hover tooltips, right-click menus, tiny drag handles, scroll wheels, text-entry fields. On a single-touch panel with no keyboard, some of that is awkward or unreachable. Big controls work fine; deep editing is genuinely useful. But this is why the macro surface exists — the native GUI is the workbench, not the stage. Text entry with no physical keyboard is an open question (§8).

6. Where it lives

Keeping auvra-core platform-free (AGENTS.md), the work splits by crate:

• auvra-clap — the gui module wraps clack’s PluginGui (negotiate → create → set_scale → set_parent → get_size/show → destroy, with resize). Register HostGui + HostTimerSupport (+ HostPosixFdSupport with the X11 path) in AuvraHost::declare_extensions (host.rs) and implement the callbacks. The GUI’d instance is a LoadedClap owned on the OS main thread (§4).
• auvra-platform — a native window-handle accessor for the SDL3 window (the NSView pointer on macOS; the X11 window id on the appliance) and a per-frame timer tick so registered plugin timers fire.
• auvra-app — an editor mode (fullscreen takeover + chrome bar), the promote-on-open flow (capture live state → main-thread build → swap slot source), and the MIXER per-slot Edit button.
• auvra-core — unchanged; it never names a window or GUI type.

7. Phasing

Deliverable	Status
Cocoa parent handle, timer callbacks, promote-on-open and modal embedded editor	Done
Resident edit graph with safe capture, patch switch and teardown ownership	Done
X11 parent handle, POSIX-fd callbacks and Linux/arm64 build/Xvfb smoke	Done
Real x86_64 Linux CLAP editor under the N100 kiosk	M6 validation
Touch-to-pointer behavior, DPI fitting and text entry on the real panel	M6 validation

8. Open questions [verify]

1. Whether each target plugin needs only timer_support or also posix_fd_support on Linux; validate with the shipping plugin matrix.
2. Single-touch → pointer emulation on the chosen X11 kiosk + capacitive panel: does the plugin editor receive usable pointer events, and how do right-click / scroll map (07 §4)?
3. Text entry for editors that need typed values, with no physical keyboard — on-screen keyboard, or intercept and provide our own field?
4. DPI / scale on the 1280×800 (or 1920×1200) panel — feed set_scale, and confirm fixed-size editors fit the editor region.

Footnotes

1. CLAP gui extension, free-audio/clap (embedded vs floating, API types, resize protocol). https://github.com/free-audio/clap/blob/main/include/clap/ext/gui.h ↩