OpenStack Summit October 2015 Tokyo

Virtualization drivers (e.g. libvirt, QEMU/KVM) are the core part of OpenStack Compute layer. An OpenStack environment is challenging to debug as is -- more so when multiple Compute nodes and thereby multiple libvirt daemons and QEMU instances are involved. A good grasp of Virtualization debugging mechanisms is vital for effective root cause analysis. To that end, libvirt and QEMU provide a rich set of debugging controls that allow us to query (or modify) the state of virtual machines in distress.

This talk focuses on providing an in-depth view of aforementioned techniques. Topics include: debugging Nova Compute process crashes; gathering specific patterns from libvirt log filters, libvirt environment variables, and systemd journal fields; live querying the VM (and QEMU) state through `virsh` and QEMU Machine Protocol (QMP) commands; tuning the libvirt daemon logging; monitoring events emitted by QEMU, etc.

Audience would include OpenStack infrastructure operators, Virtualization (libvirt/QEMU/KVM) administrators, developers, tinkerers, or any one interested in understanding the Virtualization layer in OpenStack to help equip yourself with better debugging techniques.