Performance Tests (burette)

burette is a standalone binary that runs performance benchmarks against OpenVMM using the petri test framework. It measures boot time, memory overhead, and concurrent VM scaling behavior, producing JSON reports that can be compared across builds.

Prerequisites

Linux host with /dev/kvm access (or Windows with Hyper-V)
Built openvmm, pipette, and test kernel/initrd artifacts

Build everything:

cargo build --release \
  -p burette -p openvmm -p pipette

Running Tests

Boot time

Measures launch-to-pipette-connect time using Linux direct boot:

burette run --test boot-time -o report.json

Available profiles control the VM configuration:

Profile	Description
`standard`	Full device set, serial agent, shared memory
`quiet-serial`	Like standard but suppresses kernel console
`minimal`	Pipette-as-init, minimal devices, shared memory
`minimal-private`	Same as minimal but with private memory (fastest)

# Use a specific profile
burette run --test boot-time --profile minimal -o report.json

# Custom iteration count and guest RAM
burette run --test boot-time --iterations 20 --mem-mb 1024

Memory overhead

Boots a single VM and measures host-side memory consumption for the openvmm process tree:

burette run --test memory -o memory.json

Network throughput

Measures TCP throughput (Gbps) and UDP packet rate (pps) using iperf3 with a linux_direct VM, an erofs tool image, and Consomme networking:

burette run --test network -o network.json

# Test with virtio-net instead of VMBus
burette run --test network --nic virtio-net -o network.json

Reported metrics:

memory_rss_kib — total RSS across process tree
memory_private_kib — private (non-shared) pages
memory_vmm_overhead_kib — VMM runtime overhead excluding guest RAM (Linux only; on Windows this equals private_kib)
memory_pss_kib — proportional set size (Linux only)
memory_process_count — processes in the tree

Compare shared vs. private memory overhead:

burette run --test memory --profile minimal -o shared.json
burette run --test memory --profile minimal-private -o private.json
burette compare shared.json private.json

Scale boot

Launches N VMs concurrently to measure boot time under contention and per-VM memory overhead. Default sweep: N = 1, 2, 4, 8, 16, 32, 64.

# Full geometric sweep (auto-stops at 90% host memory)
burette run --test scale-boot --mem-mb 256 -o scale.json

# Single data point
burette run --test scale-boot --vms 16 --mem-mb 256

# Custom sweep
burette run --test scale-boot --vms 1,2,4,8 --max-vms 32

Per-N metrics include scale_{N}_mean_boot_ms, scale_{N}_p99_boot_ms, scale_{N}_last_ready_ms, scale_{N}_per_vm_memory_mib, and others.

Disk I/O

Measures block I/O throughput (MiB/s) and IOPS using fio in a linux_direct VM with an erofs tool image and a data disk. Supports virtio-blk and storvsc (synthetic SCSI) backends:

# Virtio-blk with RAM-backed disk (measures virtio overhead)
burette run --test disk-io -o disk.json

# Storvsc backend
burette run --test disk-io --disk-backend storvsc -o disk.json

# File-backed disk for realistic host I/O latency
burette run --test disk-io --data-disk /tmp/test.raw --data-disk-size-gib 8

Reported metrics per backend:

fio_{backend}_seq_read_bw / fio_{backend}_seq_write_bw — sequential bandwidth (MiB/s)
fio_{backend}_rand_read_bw / fio_{backend}_rand_write_bw — random bandwidth (MiB/s)
fio_{backend}_rand_read_iops / fio_{backend}_rand_write_iops — random IOPS

By default a RAM-backed disk is used to isolate virtio/storvsc overhead without host filesystem noise. Pass --data-disk with a path on fast storage (e.g., NVMe) for end-to-end latency measurements.

Comparing Reports

burette compare baseline.json candidate.json

Prints a table of deltas and percentage changes for each metric. Optionally write the comparison to JSON:

burette compare baseline.json candidate.json -o diff.json

Remote Deployment

Package all binaries and artifacts into a self-contained tarball:

burette package -o burette_bundle.tar.gz

On the remote machine:

tar xzf burette_bundle.tar.gz
cd burette_bundle
VMM_TESTS_CONTENT_DIR=$PWD ./burette run -o report.json

The bundle includes burette, openvmm, pipette, the test kernel, and initrd — no Rust toolchain or repo checkout needed.

Running All Tests

Omit --test to run every test:

burette run -o full_report.json

JSON Report Format

Reports are JSON files with git revision info, timestamps, and per-metric statistics:

{
  "git_revision": "abc123",
  "git_commit_date": "2026-03-18T00:00:00Z",
  "date": "2026-03-18T01:00:00Z",
  "results": [
    {
      "name": "boot_time_ms",
      "unit": "ms",
      "iterations": 10,
      "mean": 126.3,
      "std_dev": 1.5,
      "min": 124.4,
      "max": 128.1
    }
  ]
}

The OpenVMM Guide