Re: Apparmor (and other) Issues

[Andrea Bolognani <abologna@redhat.com>]

On Tue, Feb 04, 2025 at 11:17:24AM +0100, Stefano Brivio wrote:
> On Tue, 4 Feb 2025 09:50:40 +0000 Andrea Bolognani <abologna@redhat.com> wrote:
> > I've skimmed the conversation trying to understand whether there's
> > anything that I need do from the libvirt side, but AFAICT no explicit
> > action has been called for so far.
>
> Not yet, because I was hoping to figure out what's going on, but I'm
> actually (almost?) stuck now. I don't think this is the same as
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1094583 by the way,

That issue can only manifest itself when upgrading from bookworm, so
if you have performed a fresh install of sid you don't need to worry
about it.

> This is really pretty simple: fresh Debian sid image, all packages
> updated to today. Then:
>
>   virt-install -d --name alpine --memory 1024 --noreboot --osinfo alpinelinux3.20 --network backend.type=passt,portForward0.proto=tcp,portForward0.range0.start=40922,portForward0.range0.to=22 --import --disk nocloud_alpine-3.21.2-x86_64-bios-tiny-r0.qcow2
>
> this works. But:
>
>   $ virsh start alpine
>   error: Failed to start domain 'alpine'
>   error: internal error: Child process (passt --one-off --socket /run/user/1000/libvirt/qemu/run/passt/1-alpine-net0.socket --pid /run/user/1000/libvirt/qemu/run/passt/1-alpine-net0-passt.pid --tcp-ports 40922:2) unexpected fatal signal 11
>
> execve() of passt is denied by AppArmor. Starting passt on its own
> (passt -f) works, instead.

I was initially very surprised to read this, but it makes sense after
spending some more time looking into it.

Normally, virt-install will perform guest creation in two phases. The
first one is the installation itself, which needs the install media
to be connected; once that's done, the OS installer will usually
initiate a reboot, and when the VM comes up again the installl media
will no longer be connected. Other changes to the configuration might
happen as well, I'm not 100% sure.

Anyway, since you've passed --noreboot above, virt-install will only
go through the first phase of installing the guest; additionally,
since you've passed --import, the VM will only be defined and the
installation will be considered done without having to start it once.

As a counter-example, see how the behavior changes when performing a
"regular" install from CDROM instead:

  $ virt-install --name alpine --memory 1024 --osinfo alpinelinux3.20 \
    --network backend.type=passt --graphics none --disk none \
    --cdrom ./alpine-virt-3.21.2-x86_64.iso
  Starting install...
  ERROR    internal error: Child process (passt --one-off
    --socket /run/user/1000/libvirt/qemu/run/passt/3-alpine-net0.socket
    --pid /run/user/1000/libvirt/qemu/run/passt/3-alpine-net0-passt.pid)
    unexpected fatal signal 11

So there's no magic to that first VM start that apparently worked
even when subsequent ones wouldn't. The VM start simply didn't happen
in the first place.

> At this point, which libvirtd (?) process should associate with which
> libvirtd profile? Once that's clear to me, I can probably debug further.
>
[...]
>
> I'm fairly sure it's libvirt, because I didn't change anything
> substantial in passt, and it's anyway the AppArmor profile for libvirtd
> that seems to be... missing? And yet it's there.

We're going back to the question of how AppArmor works for
unprivileged VMs... The short answer is that I'm not convinced it
does.

In order to make a direct comparison, I've create the very same
Alpine VM, with no network interface, both on qemu:///system and
qemu:///user.

When I start the former, the (filtered) output for aa-status goes
from

  22 profiles are in enforce mode.
     libvirtd
     libvirtd//qemu_bridge_helper
     virt-aa-helper
  25 profiles are in complain mode.
     dnsmasq//libvirt_leaseshelper
  2 processes are in enforce mode.
     /usr/sbin/libvirtd (862) libvirtd

to

  24 profiles are in enforce mode.
     libvirt-1cdfdfd3-a020-45d9-8f88-8e0ce31745c4
     libvirt-1cdfdfd3-a020-45d9-8f88-8e0ce31745c4//passt
     libvirtd
     libvirtd//qemu_bridge_helper
     virt-aa-helper
  25 profiles are in complain mode.
     dnsmasq//libvirt_leaseshelper
  3 processes are in enforce mode.
     /usr/bin/qemu-system-x86_64 (1310)
libvirt-1cdfdfd3-a020-45d9-8f88-8e0ce31745c4
     /usr/sbin/libvirtd (862) libvirtd

The additional profiles (libvirt-1cdfdfd3-a020-45d9-8f88-8e0ce31745c4
and its passt subprofile) are generated dynamically when the VM is
started and the QEMU process gets the correct one assigned to it. So
far so good.

After launching the qemu:///session VM, however, the situation is a
lot different:

  22 profiles are in enforce mode.
     libvirtd
     libvirtd//qemu_bridge_helper
     virt-aa-helper
  25 profiles are in complain mode.
     dnsmasq//libvirt_leaseshelper
  4 processes are in enforce mode.
     /usr/sbin/libvirtd (1589) libvirtd
     /usr/sbin/virtlogd (1632) libvirtd

As you can see, the per-VM profile hasn't been created, and so
obviously it couldn't be applied to the QEMU process either.

This is probably undesirable, but at the same time I'm not really
sure how it could be fixed. Creating a new profile requires dropping
a file in /etc/apparmor.d/libvirt, and the unprivileged libvirtd
intentionally doesn't have the necessary permissions to do that...

Do you have any ideas?

-- 
Andrea Bolognani / Red Hat / Virtualization