Re: Apparmor (and other) Issues

[Prafulla Giri <prafulla.giri@protonmail.com>]

On Friday, February 7th, 2025 at 3:01 PM, Stefano Brivio <sbrivio@redhat.com> wrote:

> On Fri, 07 Feb 2025 06:49:45 +0000
> Prafulla Giri prafulla.giri@protonmail.com wrote:
> 
> > On Wednesday, February 5th, 2025 at 4:01 PM, Stefano Brivio
> > sbrivio@redhat.com wrote:
> > 
> > > But the libvirt profile is not associated to the
> > > process, oops.
> > 
> > Oh, so this is what is being worked upon: that Apparmor is not making
> > the association
> 
> 
> That, I'm not sure, but at least Andrea asked openSUSE and Ubuntu
> people for comments. I just prepared (and merged) a workaround for the
> moment. You are Cc'ed on the patch. If you want to test it, you should
> add this:
> 
> # Workaround: libvirt's profile comes with a passt subprofile which includes,
> # in turn, <abstractions/passt>, and adds libvirt-specific rules on top, to
> 
> # allow passt (when started by libvirtd) to write socket and PID files in the
> # location requested by libvirtd itself, and to execute passt itself.
> #
> # However, when libvirt runs as unprivileged user, the mechanism based on
> # virt-aa-helper, designed to build per-VM profiles as guests are started,
> # doesn't work. The helper needs to create and load profiles on the fly, which
> # can't be done by unprivileged users, of course.
> #
> # As a result, libvirtd runs unconfined if guests are started by unprivileged
> # users, starting passt unconfined as well, which means that passt runs under
> # its own stand-alone profile (this one), which implies in turn that execve()
> # of /usr/bin/passt is not allowed, and socket and PID files can't be written.
> #
> # Duplicate libvirt-specific rules here as long as this is not solved in
> # libvirt's profile itself.
> /usr/bin/passt r,
> owner @{run}/user/[0-9]/libvirt/qemu/run/passt/ rw,
> owner @{run}/libvirt/qemu/passt/* rw,
> 
> to your /etc/apparmor.d/usr.bin.passt. Note that changes to AppArmor
> policy files are retained as configuration, so, if you edit it, package
> upgrades won't override things automatically. You will need to:
> 

I seem to have botched things up really good, or we're getting into more and more trouble here:

1. I have manually `make install`-ed passt (and friends).
$ passt version # I don't know what's causing the non-AVX2 thing issue
Can't run AVX2 build, using non-AVX2 version: Permission denied
passt 2025_01_21.4f2c8e7-27-gfe8b6a7
Copyright Red Hat
GNU General Public License, version 2 or later
  <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

This is my apparmor profile (for /usr/local/bin/passt):
$ cat /etc/apparmor.d/usr.bin.passt
# SPDX-License-Identifier: GPL-2.0-or-later
#
# PASST - Plug A Simple Socket Transport
#  for qemu/UNIX domain socket mode
#
# PASTA - Pack A Subtle Tap Abstraction
#  for network namespace/tap device mode
#
# contrib/apparmor/usr.bin.passt - AppArmor profile for passt(1)
#
# Copyright (c) 2022 Red Hat GmbH
# Author: Stefano Brivio <sbrivio@redhat.com>

abi <abi/3.0>,

include <tunables/global>

profile passt /usr/local/bin/passt{,.avx2} {
  include <abstractions/passt>

  # Alternatively: include <abstractions/user-tmp>
  owner /tmp/**				w,	# tap_sock_unix_open(),
						# tap_sock_unix_init(), pcap(),
						# pidfile_open(),
						# pidfile_write(),
						# logfile_init()

  owner @{HOME}/**			w,	# pcap(), pidfile_open(),
						# pidfile_write()

  # Workaround: libvirt's profile comes with a passt subprofile which includes,
  # in turn, <abstractions/passt>, and adds libvirt-specific rules on top, to
  # allow passt (when started by libvirtd) to write socket and PID files in the
  # location requested by libvirtd itself, and to execute passt itself.
  #
  # However, when libvirt runs as unprivileged user, the mechanism based on
  # virt-aa-helper, designed to build per-VM profiles as guests are started,
  # doesn't work. The helper needs to create and load profiles on the fly, which
  # can't be done by unprivileged users, of course.
  #
  # As a result, libvirtd runs unconfined if guests are started by unprivileged
  # users, starting passt unconfined as well, which means that passt runs under
  # its own stand-alone profile (this one), which implies in turn that execve()
  # of /usr/bin/passt is not allowed, and socket and PID files can't be written.
  #
  # Duplicate libvirt-specific rules here as long as this is not solved in
  # libvirt's profile itself.
  /usr/local/bin/passt r,
  owner @{run}/user/[0-9]*/libvirt/qemu/run/passt/* rw,
  owner @{run}/libvirt/qemu/passt/* rw,
}

I have restarted the machine after making these changes, for good measure.

Then:
$ virsh start --domain vm1 # throws the following error:
error: Failed to start domain 'vm1'
error: internal error: Child process (passt --one-off --socket /run/user/1000/libvirt/qemu/run/passt/1-vm1-net0.socket --pid /run/user/1000/libvirt/qemu/run/passt/1-vm1-net0-passt.pid) unexpected exit status 126: libvirt:  error : cannot execute binary passt: Permission denied

However,
$ passt --one-off --socket /run/user/1000/libvirt/qemu/run/passt/1-vm1-net0.socket --pid /run/user/1000/libvirt/qemu/run/passt/1-vm1-net0-passt.pid # this command on it's own works
Can't run AVX2 build, using non-AVX2 version: Permission denied
No interfaces with usable IPv6 routes
UNIX domain socket bound at /run/user/1000/libvirt/qemu/run/passt/1-vm1-net0.socket
Template interface: enp1s0 (IPv4)
MAC:
    host: 9a:55:9a:55:9a:55
    NAT to host 127.0.0.1: 192.168.100.1
DHCP:
    assign: 192.168.100.157
    mask: 255.255.255.0
    router: 192.168.100.1
DNS:
    192.168.100.1
DNS search list:
    .

You can now start qemu (>= 7.2, with commit 13c6be96618c):
    kvm ... -device virtio-net-pci,netdev=s -netdev stream,id=s,server=off,addr.type=unix,addr.path=/run/user/1000/libvirt/qemu/run/passt/1-vm1-net0.socket
or qrap, for earlier qemu versions:
    ./qrap 5 kvm ... -net socket,fd=5 -net nic,model=virtio

If, however, I uninstall the manually made version (with the apparmor profile back to pointing to /usr/bin/passt):

$ passt --version
passt 0.0~git20250121.4f2c8e7-1
Copyright Red Hat
GNU General Public License, version 2 or later
  <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

$ virsh start --domain vm1 
Domain 'vm1' started

So I figure it's okay after all.

I am surprised, the upstream tree does not seem to contain the recent DNS related patch that I shazamed and tested just a couple of days earlier ( b4 shazam https://archives.passt.top/passt-dev/20250203082210.2114348-1-sbrivio@redhat.com/ ). I have just emailed a reply at the patch's own thread, too: https://archives.passt.top/passt-dev/pXatirFM4Zm6ZSKXYsfeyZnHlL8JlQDKRUNtyLqbzrUOFcItX06upuk1cPYaufL1HzqHygkWvySVhawtV8zVi8Ou9qmhz_EvuQxr46UOi3Q=@protonmail.com/

Please let me know if I'm still missing (or failing to report) something. I'd hate to unknowingly be a bottle-neck by failing to test/report something.

> apt-get -o Dpkg::Options::="--force-confmiss" install --reinstall passt
> 
> > whereas SELinux is doing it's thing as it's supposed to.
> 
> 
> Right, that's because SELinux can do this:
> 
> https://selinuxproject.org/page/MultiCategorySecurity
> 
> with AppArmor, it needs to be "emulated", somehow.
> 
> > > We're just trying to make things as
> > > strict as possible, and depending on specific paths.
> > 
> > I see. I'm glad this approach of as-strict-as-possible is being taken.
> > 
> > > We'll probably need to make them a bit looser for the moment being
> > > and perhaps just allow passt, no matter who starts it, to write to
> > > /var/run/**.
> > 
> > I believe user-mode virtual machines only need access to
> > /run/user/$USER and not /var/run. Not even /run/*, but only
> > /run/user/$USER. So if that work-around is to be implemented, that
> > would be the strictest version of it: each user-started passt process
> > gets access to $XDG_RUNTIME_DIR of it's owner (and not outside of it).
> 
> 
> It depends, because if you start passt as root, socket and PID go to
> @{run}/libvirt/qemu/passt/, and if you don't, the location becomes
> @{run}/user/[0-9]/libvirt/qemu/run/passt/*.
> 
> @{run} is an AppArmor handle for /var/run and /run, I think (aren't
> they generally linked anyway?). Note that Ubuntu and openSUSE might
> use slightly different paths.
> 

/var/run deems to be a symbolic link to /run which is a tmpfs mount on my Debian Trixie machine.

> > It also seems that more and more of us use $XDG_RUNTIME_DIR in lieu
> > of /tmp in our personal shell scripts, because it kinda' feels like a
> > more private /tmp.
> 
> 
> Yeah, it is.
> 
> > Also, the `passt` update fixing DNS issue hasn't yet made it to
> > Debian Trixie, yet.
> 
> 
> I didn't release the fix yet. I merged it (upstream), but actually I
> was expecting you would give it a quick try. If I'm more confident
> about the change, I can do things faster.
> 
I thought I'd already reported it working, but, as I mentioned earlier, the patch from isn't available on the master branch yet: https://archives.passt.top/passt-dev/pXatirFM4Zm6ZSKXYsfeyZnHlL8JlQDKRUNtyLqbzrUOFcItX06upuk1cPYaufL1HzqHygkWvySVhawtV8zVi8Ou9qmhz_EvuQxr46UOi3Q=@protonmail.com/

Perhaps I'm missing something.
> > I should venture to Debian Sid, myself.
> 
> 
> It's not in Debian Sid either because I didn't make a new release of
> passt, yet.
> 
> It probably makes sense to make one next week (we release quite often,
> especially if there's one or more fixes that might be important for
> somebody, such as this one). As Debian maintainer I also update Debian
> packages within a couple of hours.
> 
> Sid to testing is usually five days of difference, look:
> 
> https://tracker.debian.org/pkg/passt/news/
> 
> --
> Stefano