AppArmor sVirt security driver for libvirt

Background
Ubuntu has been using libvirt as part of its recommended virtualization management toolkit since Ubuntu 8.04 LTS. In short, libvirt provides a set of tools and APIs for managing virtual machines (VMs) such as creating a VM, modifying its hardware configuration, starting and stopping VMs, and a whole lot more. For more information on using libvirt in Ubuntu, see http://doc.ubuntu.com/ubuntu/serverguide/C/libvirt.html.

Libvirt greatly eases the deployment and management of VMs, but due to the fact that it has traditionally been limited to using POSIX ACLs and sometimes needs to perform privileged actions, using libvirt (or any virtualization technology for that matter) can create a security risk, especially when the guest VM isn’t trusted. It is easy to imagine a bug in the hypervisor which would allow a compromised VM to modify other guests or files on the host. Considering that when using qemu:///system the guest VM process runs as a root (this is configurable in 0.7.0 and later, but still the default in Fedora and Ubuntu 9.10), it is even more important to contain what a guest can do. To address these issues, SELinux developers created a fork of libvirt, called sVirt, which when using kvm/qemu, allows libvirt to add SELinux labels to files required for the VM to run. This work was merged back into upstream libvirt in version 0.6.1, and it’s implementation, features and limitations can be seen in a blog post by Dan Walsh and an article on LWN.net. This is inspired work, and the sVirt folks did a great job implementing it by using a plugin framework so that others could create different security drivers for libvirt.

AppArmor Security Driver for Libvirt
While Ubuntu has SELinux support, by default it uses AppArmor. AppArmor is different from SELinux and some other MAC systems available on Linux in that it is path-based, allows for mixing of enforcement and complain mode profiles, uses include files to ease development, and typically has a far lower barrier to entry than other popular MAC systems. It has been an important security feature of Ubuntu since 7.10, where CUPS was confined by AppArmor in the default installation (more profiles have been added with each new release).

Since virtualization is becoming more and more prevalent, improving the security stance for libvirt users is of primary concern. It was very natural to look at adding an AppArmor security driver to libvirt, and as of libvirt 0.7.2 and Ubuntu 9.10, users have just that. In terms of supported features, the AppArmor driver should be on par with the SELinux driver, where the vast majority of libvirt functionality is supported by both drivers out of the box.

Implementation
First, the libvirtd process is confined with a lenient profile that allows the libvirt daemon to launch VMs, change into another AppArmor profile and use virt-aa-helper to manipulate AppArmor profiles. virt-aa-helper is a helper application that can add, remove, modify, load and unload AppArmor profiles in a limited and restricted way. Specifically, libvirtd is not allowed to adjust anything in /sys/kernel/security directly, or modify the profiles for the virtual machines directly. Instead, libvirtd must use virt-aa-helper, which is itself run under a very restrictive AppArmor profile. Using this architecture helps prevent any opportunities for a subverted libvirtd to change its own profile (especially useful if the libvirtd profile is adjusted to be restrictive) or modify other AppArmor profiles on the system.

Next, there are several profiles that comprise the system:

• /etc/apparmor.d/usr.sbin.libvirtd
• /etc/apparmor.d/usr.bin.virt-aa-helper
• /etc/apparmor.d/abstractions/libvirt-qemu
• /etc/apparmor.d/libvirt/TEMPLATE
• /etc/apparmor.d/libvirt/libvirt-<uuid>
• /etc/apparmor.d/libvirt/libvirt-<uuid>.files

/etc/apparmor.d/usr.sbin.libvirtd and /etc/apparmor.d/usr.bin.virt-aa-helper define the profiles for libvirtd and virt-aa-helper (note that in libvirt 0.7.2, virt-aa-helper is located in /usr/lib/libvirt/virt-aa-helper). /etc/apparmor.d/libvirt/TEMPLATE is consulted when creating a new profile when one does not already exist. /etc/apparmor.d/abstractions/libvirt-qemu is the abstraction shared by all running VMs. /etc/apparmor.d/libvirt/libvirt-<uuid> is the unique base profile for an individual VM, and /etc/apparmor.d/libvirt/libvirt-<uuid>.files contains rules for the guest-specific files required to run this individual VM.

The confinement process is as follows (assume the VM has a libvirt UUID of ‘a22e3930-d87a-584e-22b2-1d8950212bac’):

1. When libvirtd is started, it determines if it should use a security driver. If so, it checks which driver to use (eg SELinux or AppArmor). If libvirtd is confined by AppArmor, it will use the AppArmor security driver
2. When a VM is started, libvirtd decides whether to ask virt-aa-helper to create a new profile or modify an existing one. If no profile exists, libvirtd asks virt-aa-helper to generate the new base profile, in this case /etc/apparmor.d/libvirt/libvirt-a22e3930-d87a-584e-22b2-1d8950212bac, which it does based on /etc/apparmor.d/libvirt/TEMPLATE. Notice, the new profile has a profile name that is based on the guest’s UUID. Once the base profile is created, virt-aa-helper works the same for create and modify: virt-aa-helper will determine what files are required for the guest to run (eg kernel, initrd, disk, serial, etc), updates /etc/apparmor.d/libvirt/libvirt-a22e3930-d87a-584e-22b2-1d8950212bac.files, then loads the profile into the kernel.
3. libvirtd will proceed as normal at this point, until just before it forks a qemu/kvm process, it will call aa_change_profile() to transition into the profile ‘libvirt-a22e3930-d87a-584e-22b2-1d8950212bac’ (the one virt-aa-helper loaded into the kernel in the previous step)
4. When the VM is shutdown, libvirtd asks virt-aa-helper to remove the profile, and virt-aa-helper unloads the profile from the kernel

It should be noted that due to current limitations of AppArmor, only qemu:///system is confined by AppArmor. In practice, this is fine because qemu:///session is run as a normal user and does not have privileged access to the system like qemu:///system does.

Basic Usage
By default in Ubuntu 9.10, both AppArmor and the AppArmor security driver for libvirt are enabled, so users benefit from the AppArmor protection right away. To see if libvirtd is using the AppArmor security driver, do:

$ virsh capabilities
Connecting to uri: qemu:///system
<capabilities>
 <host>
  ...
  <secmodel>
    <model>apparmor</model>
    <doi>0</doi>
  </secmodel>
 </host>
 ...
</capabilities>


Next, start a VM and see if it is confined:

$ virsh start testqemu
Connecting to uri: qemu:///system
Domain testqemu started

$ virsh domuuid testqemu
Connecting to uri: qemu:///system
a22e3930-d87a-584e-22b2-1d8950212bac

$ sudo aa-status
apparmor module is loaded.
16 profiles are loaded.
16 profiles are in enforce mode.
...
  /usr/bin/virt-aa-helper
  /usr/sbin/libvirtd
  libvirt-a22e3930-d87a-584e-22b2-1d8950212bac
...
0 profiles are in complain mode.
6 processes have profiles defined.
6 processes are in enforce mode :
...
  libvirt-a22e3930-d87a-584e-22b2-1d8950212bac (6089)
...
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

$ ps ww 6089
PID TTY STAT TIME COMMAND
6089 ? R 0:00 /usr/bin/qemu-system-x86_64 -S -M pc-0.11 -no-kvm -m 64 -smp 1 -name testqemu -uuid a22e3930-d87a-584e-22b2-1d8950212bac -monitor unix:/var/run/libvirt/qemu/testqemu.monitor,server,nowait -boot c -drive file=/var/lib/libvirt/images/testqemu.img,if=ide,index=0,boot=on -drive file=,if=ide,media=cdrom,index=2 -net nic,macaddr=52:54:00:86:5b:6e,vlan=0,model=virtio,name=virtio.0 -net tap,fd=17,vlan=0,name=tap.0 -serial none -parallel none -usb -vnc 127.0.0.1:1 -k en-us -vga cirrus


Here is the unique, restrictive profile for this VM:

$ cat /etc/apparmor.d/libvirt/libvirt-a22e3930-d87a-584e-22b2-1d8950212bac
#
# This profile is for the domain whose UUID
# matches this file.
#
 
#include <tunables/global>
 
profile libvirt-a22e3930-d87a-584e-22b2-1d8950212bac {
   #include <abstractions/libvirt-qemu>
   #include <libvirt/libvirt-a22e3930-d87a-584e-22b2-1d8950212bac.files>
}

$ cat /etc/apparmor.d/libvirt/libvirt-a22e3930-d87a-584e-22b2-1d8950212bac.files
# DO NOT EDIT THIS FILE DIRECTLY. IT IS MANAGED BY LIBVIRT.
  "/var/log/libvirt/**/testqemu.log" w,
  "/var/run/libvirt/**/testqemu.monitor" rw,
  "/var/run/libvirt/**/testqemu.pid" rwk,
  "/var/lib/libvirt/images/testqemu.img" rw,


Now shut it down:

$ virsh shutdown testqemu
Connecting to uri: qemu:///system
Domain testqemu is being shutdown

$ virsh domstate testqemu
Connecting to uri: qemu:///system
shut off

$ sudo aa-status | grep 'a22e3930-d87a-584e-22b2-1d8950212bac'
[1]


Advanced Usage
In general, you can forget about AppArmor confinement and just use libvirt like normal. The guests will be isolated from each other and user-space protection for the host is provided. However, the design allows for a lot of flexibility in the system. For example:

• If you want to adjust the profile for all future, newly created VMs, adjust /etc/apparmor.d/libvirt/TEMPLATE
• If you need to adjust access controls for all VMs, new or existing, adjust /etc/apparmor.d/abstractions/libvirt-qemu
• If you need to adjust access controls for a single guest, adjust /etc/apparmor.d/libvirt-<uuid>, where <uuid> is the UUID of the guest
• To disable the driver, either adjust /etc/libvirt/qemu.conf to have ‘security_driver = “none”‘ or remove the AppArmor profile for libvirtd from the kernel and restart libvirtd

Of course, you can also adjust the profiles for libvirtd and virt-aa-helper if desired. All the files are simple text files. See AppArmor for more information on using AppArmor in general.

Limitations and the Future
While the sVirt framework provides good guest isolation and user-space host protection, the framework does not provide protection against in-kernel attacks (eg, where a guest process is able to access the host kernel memory). The AppArrmor security driver as included in Ubuntu 9.10 also does not handle access to host devices as well as it could. Allowing a guest to access a local pci device or USB disk is a potentially dangerous operation anyway, and the driver will block this access by default. Users can work around this by adjusting the base profile for the guest.

There are few missing features in the sVirt model, such as labeling state files. The AppArmor driver also needs to better support host devices. Once AppArmor provides the ability for regular users to define profiles, then qemu:///session can be properly supported. Finally, it will be great when distributions take advantage of libvirt’s recently added ability to run guests as non-root when using qemu:///system (while the sVirt framework largely mitigates this risk, it is good to have security in depth).

Summary
While cloud computing feels like it is talked about everywhere and virtualization becoming even more important in the data center, leveraging technologies like libvirt and AppArmor is a must. Virtualization removes the traditional barriers afforded to stand-alone computers, thus increasing the attack surface for hostile users and compromised guests. By using the sVirt framework in libvirt, and in particular AppArmor on Ubuntu 9.10, administrators can better defend themselves against virtualization-specific attacks. Have fun and be safe!

More Information
http://libvirt.org/
https://wiki.ubuntu.com/AppArmor
https://wiki.ubuntu.com/SecurityTeam/Specifications/AppArmorLibvirtProfile