gRPC / ttrpc
To enable a gRPC or ttrpc management interface, pass --rpc. This spawns an
OpenVMM process acting as an RPC server on the given Unix socket:
--rpc path=/path/to/openvmm.sock[,transport=<TRANSPORT>]
transport selects which wire protocol the server accepts:
auto(default) — auto-detect ttrpc vs. gRPC per connectionttrpc— accept ttrpc clients onlygrpc— accept gRPC clients only
For example, to accept ttrpc clients only:
--rpc path=/path/to/openvmm.sock,transport=ttrpc
Here is a list of supported RPCs:
The API continues to evolve, and compatibility between releases is not
guaranteed. The vmservice.proto file is the authoritative API definition.
The list below summarizes the available RPCs; some definitions may be added
before their implementation is connected end to end.
- CreateVM
- TeardownVM
- PauseVM
- ResumeVM
- SaveVM
- RestoreVM
- ShutdownVM
- WaitVM
- CapabilitiesVM
- PropertiesVM
- ModifyResource
- AddPcieDevice
- RemovePcieDevice
- AddVpciDevice
- RemoveVpciDevice
- Quit
Snapshots (SaveVM and RestoreVM)
SaveVM is a compound operation that pauses the VM and writes manifest.bin,
state.bin, and memory.bin to the requested directory. The VM must use the
MemoryConfig.backing_file_path setting. The currently supported
implementation hard-links that backing file into the snapshot, so after a
successful save the VM can't be resumed and should be torn down. Each save
generates a unique snapshot ID (a GUID string) that is stored in
manifest.bin and returned in SaveVMResponse.snapshot_id.
Cancelling SaveVM only stops the caller from waiting; the save may still
complete.
RestoreVM creates a VM from a snapshot directory and a compatible
VMConfig. Device configuration is not stored in the snapshot and must be
supplied by the caller. memory.bin is used directly as mutable guest memory,
so MemoryConfig.backing_file_path must not be set, and a snapshot can be
restored only once. If expected_snapshot_id is provided, the restore fails
unless it matches the snapshot ID in manifest.bin. An empty value is treated
as not provided, so the ID isn't checked. The restored snapshot's
ID is returned in RestoreVMResponse.snapshot_id. Set resume to start the
restored VM immediately; otherwise it is created paused.
Virtio-console file output
VmService can write a virtio-console stream directly to a host file. Set the
console's SerialBackend to SerialFile:
pcie {
root_complexes {
name: "rc0"
root_ports {
name: "console"
attached { device { virtio { console {
backend { file {
path: "path/to/console.log"
append: false
} }
} } } }
}
}
}
The file is opened while CreateVM processes the device configuration, so an
invalid path or permission failure causes CreateVM to fail. With append
unset or false, OpenVMM creates or truncates the file. With append: true,
OpenVMM preserves existing contents and appends new guest output.
The backend remains open for the console lifetime. OpenVMM does not rotate or
limit the file, so the management client owns retention. SerialRelay remains
available when the client needs an interactive or externally managed stream.
The backend writes synchronously to the file, without an intermediate socket,
named pipe, or background copy task. Successful writes complete before returning
to the serial device, so there is no background output to drain during teardown.
A slow or stalled file write can block the calling device thread. After
TeardownVM or Quit completes, the management client can read the completed
output. Writes do not force a disk synchronization and do not guarantee that the
output survives a host crash or power failure.
If a write fails, OpenVMM logs the error and disconnects the file backend.
Further console output is discarded; the backend does not automatically retry
or reopen the file.
Passing handles on Windows
Some API fields reference OS handles by name. For example,
VirtioFs.WindowsOptions.user_token_name names an access token that virtio-fs
impersonates for host file access.
Clients register these handles over the same socket using the
resource-passing protocol. Because Windows Unix sockets cannot carry handles,
the client duplicates each handle into the OpenVMM process with
DuplicateHandle, which requires PROCESS_DUP_HANDLE access to it, and sends
the handle value under a name. OpenVMM makes its own duplicate; the client then
closes its copy in OpenVMM.
Graceful guest shutdown
ShutdownVM requests a cooperative guest power-off, non-forced by default.
Success means IC acknowledgement or ACPI event injection, not that the guest has halted. Neither mechanism falls back to the other or forcibly powers off the VM.
See ShutdownVM and ShutdownVMRequest in vmservice.proto for prerequisites,
request fields, shutdown lifecycle.
VPCI hotplug
AddVpciDevice dynamically exposes a PCI device to VTL0 over Hyper-V VPCI.
The VM must have Hyper-V enlightenments and VMBus enabled, and the host
hypervisor backend must support virtual devices. The caller supplies the
guest-visible instance ID in AddVpciDeviceRequest.instance_id and uses the
same ID for RemoveVpciDevice. The response is empty. Removing an unknown
or previously removed instance ID returns an error.
Unlike AddPcieDevice, VPCI does not require a root complex or a predeclared
hotplug-capable PCIe port. AddPcieDevice remains available when standard PCIe
hotplug semantics or a non-VPCI host backend is required.
Hvsocket relay listeners
Use ModifyResource with hvsocket_listener to relay a host Unix socket to a
guest hvsocket service on VTL0. VMBus and hvsocket support must be available.
Specify either a service_id GUID or a vsock_port; both identify the same
service when the GUID is the vsock port's hvsocket mapping.
For example, this protobuf text-format ModifyResourceRequest adds a listener:
type: ADD
hvsocket_listener {
path: "path/to/service.sock"
vsock_port: 50001
}
Clients send application data immediately. These listeners do not require a
hybrid-vsock CONNECT request and do not send an OK response.
Only one listener per service is supported. ADD fails if the path already
exists, including a stale socket file; it does not replace or remove that path.
The caller owns socket-file cleanup, including after a failed ADD that
successfully bound the path before failing to register the listener.
To remove a listener, send REMOVE with the same service selector; path is
ignored. Removal waits for the listening socket to close, but established relay
connections remain active. Removing an unknown service returns an error.
Remove the socket file yourself before reusing its path.
Consomme NIC modifications
Use ModifyResource with type: UPDATE and nic_config.nic_id to modify
an existing Consomme NIC. Select nic_config.consomme and supply either
ports, add_dns_record, or create_virtual_address. DNS and virtual-address
operations cannot be combined with port modifications or used at VM creation.
They are not supported for other NIC backends.
For example, this protobuf text-format request adds a static DNS A record:
type: UPDATE
nic_config {
nic_id: "11111111-1111-1111-1111-111111111111"
consomme {
add_dns_record { name: "host.example" address: "192.0.2.1" }
}
}
To allocate a guest-visible alias for a host IPv4 or IPv6 address, use
create_virtual_address { destination: "127.0.0.1" } instead. The allocated
address is returned in
ModifyResourceResponse.virtual_address.virtual_address. Repeating the
operation for the same destination returns the same alias.
DNS and port changes return a ModifyResourceResponse without a result.
Port forwarding retains its existing semantics: UPDATE binds the supplied
ports and REMOVE unbinds them. DNS and virtual-address operations only
support UPDATE.
VFIO and accelerated SMMU
On Linux, declare named host iommufd contexts in VMConfig.iommufds, then
reference a context with VfioDevice.iommufd_id. The server opens
/dev/iommu and uses VFIO cdev assignment. Devices referencing the same ID
share the context and its DMA address space. The host device must already
be bound to vfio-pci.
Context IDs must be non-empty and unique within the VM. An empty or unknown
device reference is an error, as is unavailable iommufd support. Omitting
iommufd_id selects legacy VFIO group/container assignment; failures on the
iommufd path do not fall back to legacy assignment.
Contexts are declared at VM creation and retained until teardown, even when
unused or after their last device is removed. AddPcieDevice uses the same
VfioDevice message and can reference any declared context. There is no RPC
to add or remove contexts, and no client file descriptor is needed.
For AArch64 guests, configure a guest-visible SMMUv3 with
PcieRootComplex.iommu.smmu. This is separate from the host iommufd context.
The following protobuf text-format fragment shows PCIe configuration for an
accelerated SMMU and an assigned device; add the VM's boot, memory, and
processor configuration to form a complete VMConfig:
iommufds { id: "iommu0" }
pcie {
root_complexes {
name: "rc0"
end_bus: 255
low_mmio: 67108864
high_mmio: 1073741824
iommu { smmu { accel: true } }
root_ports {
name: "rp0"
hotplug: true
attached {
device {
vfio {
host_pci_address: "0000:01:00.0"
iommufd_id: "iommu0"
}
}
}
}
}
}
accel: true enables hardware nested translation and requires ACPI, a
nesting-capable host SMMUv3, and hypervisor support for the assigned-device
MSI IOVA reservation. VFIO devices behind the SMMU must use a single shared
iommufd context. With accel false, the SMMU uses software translation and
does not support VFIO assignment behind it. Omitting iommu leaves the root
complex without a guest-visible IOMMU.
SmmuConfig.oas_bits selects a fixed output address size in bits. Omit it for
the CLI's oas=auto policy: initially 48 bits, adopting the physical SMMU's
width when an accelerated device is available during VM construction. Later
device attachments must support the resulting advertised width. A fixed width
cannot exceed the physical SMMU's width with acceleration.
SmmuConfig.ssid_bits selects a fixed SubstreamID width from 0 to 20.
Explicit zero disables substreams. Omitting the field selects ssidsize=auto:
zero in software mode, or the cold-plug host SMMU's width under acceleration.
Later device attachments must support the resulting advertised width. A fixed
width must not exceed the host's width under acceleration.
SMMU configuration cannot be changed at runtime. See the CLI reference and Arm SMMUv3 for platform requirements.
Extending the API
When editing vmservice.proto, follow the
Protocol Buffers enum conventions.