Debugging OpenHCL

OpenHCL provides several debugging tools for investigating issues at the user-mode and kernel level. See ohcldiag-dev for the diagnostic client and Tracing for serial and event log tracing.

On-demand memory dumps

Use ohcldiag-dev dump to capture a live user-mode memory dump of the OpenHCL process at any time. The dump is an ELF core file that you can analyze with lldb, gdb, or rust-lldb. See the ohcldiag-dev page for the full command reference.

User-mode crash dumps

When an OpenHCL user-mode process crashes, a crash dump is automatically generated via the underhill-crash infrastructure and sent to the host over VMBus. On Windows hosts, these dumps are collected by Windows Error Reporting (WER). Use lldb or gdb to analyze the resulting ELF core dump.

Kernel crash dumps

Kernel-mode crash dumps (kdump) are not currently supported in OpenHCL. The OpenHCL kernel does not have CONFIG_KDUMP or CONFIG_KEXEC compiled in. If the kernel panics, no dump is generated. The only diagnostic output is COM3 serial (if enabled), which captures the panic message in real time. If the diagnostic service was running before the panic, ohcldiag-dev may have ring buffer messages up to that point, but it cannot capture the panic itself since the service is terminated by the panic.

For debugging kernel-level issues, the best approach is to enable serial output via COM3 (see below) — it captures output from the very first instruction of kernel boot.

Getting OpenHCL kernel logs (COM3 vs ohcldiag-dev)

Two methods exist for capturing OpenHCL kernel (kmsg) output:

COM3 serial uses direct UART I/O — it streams output from the very first instruction of OpenHCL boot in real time.

ohcldiag-dev connects over vsock to the diagnostic service, which reads /dev/kmsg. Because /dev/kmsg preserves the kernel ring buffer, early boot messages are replayed when you connect — you get them even if you connect late. However, ohcldiag-dev only works if the diagnostic service successfully starts.

Boot phaseCOM3 serialohcldiag-dev
Very early kernel (entry → memory setup)✅ live✅ replayed from ring buffer
Device initialization (VMBus, etc.)✅ live✅ replayed from ring buffer
Kernel panic before userspace✅ live❌ service never starts
Boot hang (kernel stuck)✅ live❌ service never starts
After diagnostic service starts✅ live✅ live

For most development, ohcldiag-dev is sufficient — boot succeeds and you get logs. COM3 is essential for debugging early boot failures, kernel panics, and init crashes.

Enabling COM3 on Hyper-V

COM3 support requires a host OS build that includes the EnableAdditionalComPorts code path. This was added in Windows 11 26H1 (build 28000+, Insider Canary channel). It is not available on Windows 11 24H2, 25H2, or Windows Server 2025.

To enable COM3 on a supported build:

# Enable additional COM ports (requires reboot or VMMS restart)
reg add "HKLM\Software\Microsoft\Windows NT\CurrentVersion\Virtualization" /v EnableAdditionalComPorts /t REG_DWORD /d 1 /f

# Attach COM3 to a named pipe for a VM
Set-VMComPort -VMName $VmName -Number 3 -Path "\\.\pipe\openhcl-com3"

# Read the serial output
hvc serial -c -p 3 -r $VmName

Note

The flowey test runner (install_vmm_tests_deps) sets this registry key automatically when running VMM tests. If you run cargo xflowey to execute tests, you'll be prompted to allow the registry change.

We recommend running a Windows 11 26H1 Insider flight (Canary channel, build 28000+) on your development machine, if it is available for your device. This gives you COM3 support via the registry key above, plus access to the latest Hyper-V features. This matches the OS used on the project's self-hosted CI runners.

Note that 26H1 Insider builds may not be available for all hardware — see What to know about Windows 11 version 26H1 for details and the Windows Insider Flight Hub for availability.

If you're on Windows 11 24H2/25H2 (builds 26100/26200), COM3 is not available via the registry key. Use ohcldiag-dev for kernel logs instead.

ARM64 limitation

On Hyper-V, additional serial ports (COM3+) are not supported on ARM64. The Hyper-V serial device for ARM64 does not support ports beyond COM1 and COM2. On ARM64 hosts, use ohcldiag-dev for OpenHCL kernel logs.

This limitation is Hyper-V-specific — when running OpenVMM directly (without Hyper-V), ARM64 serial output works via PL011 UART.