Re: Apparmor (and other) Issues

[Stefano Brivio <sbrivio@redhat.com>]

On Tue, 4 Feb 2025 15:50:43 +0000
Andrea Bolognani <abologna@redhat.com> wrote:

> On Tue, Feb 04, 2025 at 11:17:24AM +0100, Stefano Brivio wrote:
> > On Tue, 4 Feb 2025 09:50:40 +0000 Andrea Bolognani <abologna@redhat.com> wrote:  
> > > I've skimmed the conversation trying to understand whether there's
> > > anything that I need do from the libvirt side, but AFAICT no explicit
> > > action has been called for so far.  
> >
> > Not yet, because I was hoping to figure out what's going on, but I'm
> > actually (almost?) stuck now. I don't think this is the same as
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1094583 by the way,  
> 
> That issue can only manifest itself when upgrading from bookworm, so
> if you have performed a fresh install of sid you don't need to worry
> about it.
> 
> > This is really pretty simple: fresh Debian sid image, all packages
> > updated to today. Then:
> >
> >   virt-install -d --name alpine --memory 1024 --noreboot --osinfo alpinelinux3.20 --network backend.type=passt,portForward0.proto=tcp,portForward0.range0.start=40922,portForward0.range0.to=22 --import --disk nocloud_alpine-3.21.2-x86_64-bios-tiny-r0.qcow2
> >
> > this works. But:
> >
> >   $ virsh start alpine
> >   error: Failed to start domain 'alpine'
> >   error: internal error: Child process (passt --one-off --socket /run/user/1000/libvirt/qemu/run/passt/1-alpine-net0.socket --pid /run/user/1000/libvirt/qemu/run/passt/1-alpine-net0-passt.pid --tcp-ports 40922:2) unexpected fatal signal 11
> >
> > execve() of passt is denied by AppArmor. Starting passt on its own
> > (passt -f) works, instead.  
> 
> I was initially very surprised to read this, but it makes sense after
> spending some more time looking into it.

Ah, sure, sorry! I didn't mean to waste your time with that, it was
clear to me that passt doesn't start there... I just wanted to show the
setup.

> [...]
>
> So there's no magic to that first VM start that apparently worked
> even when subsequent ones wouldn't. The VM start simply didn't happen
> in the first place.
> 
> > At this point, which libvirtd (?) process should associate with which
> > libvirtd profile? Once that's clear to me, I can probably debug further.
> >  
> [...]
> >
> > I'm fairly sure it's libvirt, because I didn't change anything
> > substantial in passt, and it's anyway the AppArmor profile for libvirtd
> > that seems to be... missing? And yet it's there.  
> 
> We're going back to the question of how AppArmor works for
> unprivileged VMs... The short answer is that I'm not convinced it
> does.
> 
> In order to make a direct comparison, I've create the very same
> Alpine VM, with no network interface, both on qemu:///system and
> qemu:///user.
> 
> When I start the former, the (filtered) output for aa-status goes
> from
> 
>   22 profiles are in enforce mode.
>      libvirtd
>      libvirtd//qemu_bridge_helper
>      virt-aa-helper
>   25 profiles are in complain mode.
>      dnsmasq//libvirt_leaseshelper
>   2 processes are in enforce mode.
>      /usr/sbin/libvirtd (862) libvirtd
> 
> to
> 
>   24 profiles are in enforce mode.
>      libvirt-1cdfdfd3-a020-45d9-8f88-8e0ce31745c4
>      libvirt-1cdfdfd3-a020-45d9-8f88-8e0ce31745c4//passt
>      libvirtd
>      libvirtd//qemu_bridge_helper
>      virt-aa-helper
>   25 profiles are in complain mode.
>      dnsmasq//libvirt_leaseshelper
>   3 processes are in enforce mode.
>      /usr/bin/qemu-system-x86_64 (1310)
> libvirt-1cdfdfd3-a020-45d9-8f88-8e0ce31745c4
>      /usr/sbin/libvirtd (862) libvirtd
> 
> The additional profiles (libvirt-1cdfdfd3-a020-45d9-8f88-8e0ce31745c4
> and its passt subprofile) are generated dynamically when the VM is
> started and the QEMU process gets the correct one assigned to it. So
> far so good.

Why do we need additional profiles, I wonder? Are we trying to
replicate the MCS (Multi-Category Security) stuff from SELinux?

> After launching the qemu:///session VM, however, the situation is a
> lot different:
> 
>   22 profiles are in enforce mode.
>      libvirtd
>      libvirtd//qemu_bridge_helper
>      virt-aa-helper
>   25 profiles are in complain mode.
>      dnsmasq//libvirt_leaseshelper
>   4 processes are in enforce mode.
>      /usr/sbin/libvirtd (1589) libvirtd
>      /usr/sbin/virtlogd (1632) libvirtd
> 
> As you can see, the per-VM profile hasn't been created, and so
> obviously it couldn't be applied to the QEMU process either.

Oh, now I understand the whole "template" thing finally! And now that
you mention virt-aa-helper (I wasn't aware of that) I also found:

  https://gitlab.com/apparmor/apparmor/-/wikis/Libvirt

Yes, of course, that won't work unless you're root.

> This is probably undesirable, but at the same time I'm not really
> sure how it could be fixed.

If I understood the purpose correctly: implementing equivalent MCS
functionality in AppArmor would probably be the only "complete" fix. But
for the moment (or more realistically):

> Creating a new profile requires dropping
> a file in /etc/apparmor.d/libvirt, and the unprivileged libvirtd
> intentionally doesn't have the necessary permissions to do that...

Profiles could reside in other places too, but they won't be associated
"just like that". A privileged helper could... help. But if users can
edit their own profiles in their homes, that defies a bit the point of
AppArmor profiles altogether.

> Do you have any ideas?

Probably you should ask somebody more familiar (openSUSE/Ubuntu
maintainers of libvirt, or the authors of the virt-aa-helper thing?),
but a couple of quick ideas:

1. user-specific subprofiles could be added at adduser time (is there a
   hook?) and libvirtd could use aa_change_hat(2) to select the
   appropriate one based on UID

2. (most reasonable I think) don't use per-VM profiles for the rootless
   case. Define a single "libvirt-user" (or "libvirt-session") profile
   and use that. We could copy it from the existing ones I suppose.

I think that MCS-equivalent functionality is anyway much less critical
if guests are started as unprivileged users because you have a strong
separation given by the fact that guests are started as different users.

If all the guests are started as root, and multiple users have access
to that facility, user separation is much more seriously endangered,
hence a stronger need for per-VM profiles.

And, stating the obvious, it's counterproductive to force users to run
guests as root because we can't have that kind of functionality.

By the way, I recently happened to introduce an AppArmor profile for
guestfs-tools:

  https://salsa.debian.org/libvirt-team/guestfs-tools/-/merge_requests/1

there are no separate profiles, there.

I can try to work on this but I'm really a bit too busy right now. If
you can, great, otherwise, let me know. On the other hand this feels
pretty urgent. :(

-- 
Stefano