From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=MVEGYTY5; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by passt.top (Postfix) with ESMTPS id C157B5A0619 for ; Thu, 09 Oct 2025 01:02:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1759964574; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Eg2EKjQFP0W3co3pWcg8vlwhtI6J/ilQnWt6lKccz9g=; b=MVEGYTY5enVD/I4KBkFrGvJHUtoNR0CCLhIh6UnE9TP/orfUZN+fJUdGr8VFqTXGY08Bnf h+/d0XziS57k28eUvy+5Z0sT3LRHEUu982DHost6vA0LnBdSQNI4JgqLm8AcksyHjaTGHe GDxlG/vswafYQp3dhrejF2oPb/TqNiE= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-76-Fjlubm-DOUa40LWeKdFwXw-1; Wed, 08 Oct 2025 19:02:53 -0400 X-MC-Unique: Fjlubm-DOUa40LWeKdFwXw-1 X-Mimecast-MFC-AGG-ID: Fjlubm-DOUa40LWeKdFwXw_1759964572 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-46e4c8fa2b1so1799325e9.0 for ; Wed, 08 Oct 2025 16:02:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759964572; x=1760569372; h=content-transfer-encoding:mime-version:organization:references :in-reply-to:message-id:subject:cc:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=di49GYmqXAOh+zPj0WH4MVYNAmAU3gu2YX3z+L9IMfY=; b=iNHV+VcFzqdiBe4wiZZ+nL+9W8j80nCtovltin2rzXExLjFmXdBWjlaK5Cq/HLh6i4 H1k5qngJNyizsqLSaFYTfXuEwGwqaX043e/SXnZDPLIpw4i18E3kCt6vtKqJg2X7dsT3 mUxyMYw/gWSZ9lvCQiKpxb68Si4HffPR6alM64ucpTqipe8INFaVA9KjQ5okbLpt1N06 iDFGGlhgFTiWgekXAxTEJkiUepbyic+aJnKgbuoOWa/g/K/FK/eWQj2AIsEdYzprdOlQ 758BKunPHEWWU0aFF/EQxuyjVr/jCz2/HgS6HupylA5zeFZONo774+ZsPoD4h5JbXs1V 2Axg== X-Gm-Message-State: AOJu0YznefChNI5yLctq8puhPnH1cOFsEhRhRRzMimj2w6ys99vGpnbf nC5luPgPPx8K4kUM8MvnFCQJ4KuiqmVBc0xSV3L7gzQLgM/EUR+Fyf5r2LjrIhFSKonSFLqHOi4 CV09FHydblluwp4UyO+6hQoYuNOg6wcbcG5pKt7nb1swDyGtrPHOpaw== X-Gm-Gg: ASbGncs9jgenYS9bMBh0MFBLrToXovbkyN9KM0iwVbRLpDHYDCmK9um7xG39LfDR5kN qCJnazbOD2vj6YzVqF67K2KcVide3kzFBHddB63n8owY3Jrb6u6hpH3wDZN/B1kE2/+noXpuYjU uZHI2JKq4XyE1YhdFN8RW+Xc2b6ERII4dPCYiDu2kkEHtTccI/1HTNSNVopplC7Xp4C8M82AECr cvWUmsT1NA9bztYpwclkCKxLVzdLPPu9UwZE3ODe3p7tbmY0p7MT9EuW3l33J4zt3BOfDn+3R/f qf9m9HdrJFVwRhFrZBgoaeE4sPSss2npunFLcBOJv2+OGHHC4NI5X0/hJ3Oe6bkTv6CBVLOo7A= = X-Received: by 2002:a05:600c:19c6:b0:45b:868e:7f7f with SMTP id 5b1f17b1804b1-46fa9aef7edmr36058715e9.17.1759964571497; Wed, 08 Oct 2025 16:02:51 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH2A2Yrimoo2sxJLr6hC/OUXDg8D4ykRRzbRWDwsUd5mvm72KNvRN0HelD6umG9j1jUs6O29Q== X-Received: by 2002:a05:600c:19c6:b0:45b:868e:7f7f with SMTP id 5b1f17b1804b1-46fa9aef7edmr36058465e9.17.1759964570631; Wed, 08 Oct 2025 16:02:50 -0700 (PDT) Received: from maya.myfinge.rs (ifcgrfdd.trafficplex.cloud. [176.103.220.4]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-46faf10563dsm17923375e9.3.2025.10.08.16.02.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Oct 2025 16:02:49 -0700 (PDT) Date: Thu, 9 Oct 2025 01:02:48 +0200 From: Stefano Brivio To: David Gibson Subject: Re: [PATCH 3/3] test: Re-implement pasta NDP tests using tunbridge & exeter Message-ID: <20251009010248.1ebc1a50@elisabeth> In-Reply-To: References: <20251002075708.461931-1-david@gibson.dropbear.id.au> <20251002075708.461931-4-david@gibson.dropbear.id.au> <20251007220110.3c8bf21c@elisabeth> Organization: Red Hat X-Mailer: Claws Mail 4.2.0 (GTK 3.24.49; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: sMJBZzjRab9t3Bcv-GCXvEfb18goK49Pi12yqWgbcg4_1759964572 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Message-ID-Hash: JGRVKX3NFRKVK5WQCVBPAWCZ4NVK7ALP X-Message-ID-Hash: JGRVKX3NFRKVK5WQCVBPAWCZ4NVK7ALP X-MailFrom: sbrivio@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Wed, 8 Oct 2025 13:32:27 +1100 David Gibson wrote: > On Tue, Oct 07, 2025 at 10:01:10PM +0200, Stefano Brivio wrote: > > On Thu, 2 Oct 2025 17:57:08 +1000 > > David Gibson wrote: > > =20 > > > Convert the pasta NDP tests from shell and our own DSL to Python usin= g > > > the exeter test protocol and tunbridge network simulation library. > > >=20 > > > Signed-off-by: David Gibson > > > --- > > > test/Makefile | 2 +- > > > test/pasta/dhcp | 5 ++++ > > > test/pasta/ndp.py | 59 ++++++++++++++++++++++++++++++++++++++++= ++ > > > test/run | 6 +++-- > > > test/tasst/__init__.py | 4 +++ > > > test/tasst/pasta.py | 40 ++++++++++++++++++++++++++++ > > > 6 files changed, 113 insertions(+), 3 deletions(-) > > > create mode 100755 test/pasta/ndp.py > > > create mode 100644 test/tasst/pasta.py > > >=20 > > > diff --git a/test/Makefile b/test/Makefile > > > index f66c7e7e..95e3d75e 100644 > > > --- a/test/Makefile > > > +++ b/test/Makefile > > > @@ -67,7 +67,7 @@ ASSETS =3D $(DOWNLOAD_ASSETS) $(LOCAL_ASSETS) > > > =20 > > > EXETER_SH =3D smoke/smoke.sh build/static_checkers.sh > > > EXETER_PYPATH =3D exeter/py3:tunbridge/:. > > > -EXETER_PYTHON =3D smoke/smoke.py build/build.py > > > +EXETER_PYTHON =3D smoke/smoke.py build/build.py pasta/ndp.py > > > EXETER_BATS =3D $(EXETER_SH:%=3D%.bats) $(EXETER_PYTHON:%=3D%.bats) > > > BATS_FILES =3D $(EXETER_BATS) \ > > > =09podman/test/system/505-networking-pasta.bats > > > diff --git a/test/pasta/dhcp b/test/pasta/dhcp > > > index e1c66be6..61279fbf 100644 > > > --- a/test/pasta/dhcp > > > +++ b/test/pasta/dhcp > > > @@ -18,6 +18,11 @@ test=09Interface name > > > nsout=09IFNAME ip -j link show | jq -rM '.[] | select(.link_type =3D= =3D "ether").ifname' > > > check=09[ -n "__IFNAME__" ] > > > =20 > > > +# Bring up the interface > > > +ns=09ip link set dev __IFNAME__ up > > > +# Wait for SLAAC & DAD to complete > > > +ns=09while ! ip -j -6 addr show dev __IFNAME__ | jq -e '.[].addr_inf= o.[] | select(.protocol =3D=3D "kernel_ra")'; do sleep 0.1; done > > > + > > > test=09DHCP: address > > > ns=09/sbin/dhclient -4 --no-pid __IFNAME__ > > > nsout=09ADDR ip -j -4 addr show|jq -rM '.[] | select(.ifname =3D=3D = "__IFNAME__").addr_info[0].local' > > > diff --git a/test/pasta/ndp.py b/test/pasta/ndp.py > > > new file mode 100755 > > > index 00000000..8c7ce31e > > > --- /dev/null > > > +++ b/test/pasta/ndp.py > > > @@ -0,0 +1,59 @@ > > > +#! /usr/bin/env python3 > > > +# > > > +# SPDX-License-Identifier: GPL-2.0-or-later > > > +# > > > +# test/pasta/ndp.py - pasta NDP functionality > > > +# > > > +# Copyright Red Hat > > > +# Author: David Gibson > > > + > > > +import contextlib > > > +import dataclasses > > > +from typing import Iterator > > > + > > > +import exeter > > > +import tunbridge > > > +import tasst > > > + > > > + > > > +@dataclasses.dataclass > > > +class UnconfiguredScenario(exeter.Scenario): > > > + """Tests for a pasta instance without --config-net""" > > > + > > > + host: tunbridge.Site > > > + guest: tunbridge.Site > > > + ifname: str > > > + addr6: tunbridge.ip.AddrMask6 > > > + gw6: tunbridge.ip.Addr6 =20 > >=20 > > Until this point, it looks like stuff I can happily copy and paste, > > and grasp, even. But then: > > =20 > > > + @exeter.scenariotest > > > + def test_ifname(self) -> None: > > > + ifs =3D tunbridge.ip.ifs(self.guest) > > > + exeter.assert_eq(set(ifs), {'lo', self.ifname}) =20 > >=20 > > ...why does a "Scenario" have a .ifname? =20 >=20 > Yeah, the readability of the Scenario mechanism was something I was > particularly concerned about. I think the concept is valuable, but > I'm very open to different ways of naming or organising it, if we can > up with something better. >From the description you give below, the name seems to fit. > A "Scenario" (specifically a subclass of exeter.Scenario) is a group > of tests with a common set of parameters. In this case > UnconfiguredScenario is a bunch of tests about the behaviour of pasta > without --config-net. Each of those tests has access to the host and > guest sites, the expected interface name, address and gateway in the > guest - that is, the contents of an UncofiguredScenario instance. I'm not sure if I understand this correctly, but if each guest has a single interface, that sounds a bit limiting. Actually, I think any abstraction that doesn't offer arbitrary sets of (and relationships between) the objects shown via netlink (or, at least, namespaces, links, routes, addresses, neighbours) might be limiting and not generic enough. > That instance describes a real (simulated) environment in which we can > run those tests. >=20 > You use this by supplying a function which sets things up, then yields > an UnconfiguredScenario instance describing what it set up. exeter > will run all of the UnconfiguredScenario tests on the environment the > setup function created, each one as a separate test case. This part is now clear to me, and I think it's not complicated to grasp the concept vaguely but enough to copy, paste, and modify code doing this. It would be even better to hide this entirely, because "yielding a scenario" is a Python thing. In general, there's an imperative part in all this (bordering functional programming, but still, not descriptive) which I struggle to see as beneficial. Here the tasks at hand are, roughly: 1. represent two network namespaces, with two interfaces each (loopback and non-loopback), with pasta connecting one of the interfaces of the inner one 2. bring up one of the interfaces 3. compare addresses ...and doing 1. like that is simply not... intuitive, I think. > Usually, there are multiple ways to set up a suitable enviroment: > running pasta with an existing guest ns vs. pasta creating the ns is a > simple example. You can create different setup functions for each of > those, and re-use all the tests in the Scenario against each of those > setups. >=20 > > > + > > > + @tunbridge.ndp.NdpScenario.subscenario > > > + def test_ndp(self) -> tunbridge.ndp.NdpScenario: > > > + tunbridge.ip.ifup(self.guest, self.ifname) =20 > >=20 > > This raises the question of how much of tunbridge one needs to know to > > be able to write a basic test. Why is ifup() in 'ip'? I thought it > > would be more of a "link" thing. =20 >=20 > Finding misleading names is a big reason for seeking early feedback. > There's kind of a reason for ifup to be in ip: it optionally takes IP > addresses to configure on the interface. But... there's no inherent > reason it couldn't take other sorts of network address too, so I'll > look into moving that into a "link" module or something like it. I think sticking to netlink objects would make this a bit more familiar, if possible. > > I admit I haven't had time to browse tunbridge recently, I'm just > > looking at this series right now. =20 >=20 > That's fine. At some point it would be good to have you look at > tunbridge too, but reading this series _without_ reading tunbridge is > a very useful perspective at this stage. >=20 > > =20 > > > + return tunbridge.ndp.NdpScenario(client=3Dself.guest, > > > + ifname=3Dself.ifname, > > > + network=3Dself.addr6.networ= k, > > > + gateway=3Dself.gw6) =20 > >=20 > > This makes sense to me. =20 >=20 > Ok, good. The Scenario stuff might not be as impenetrable as I > feared. Here I was simply commenting on the fact that I intuitively understand those arguments and how they belong to the scenario, not on the Scenario abstraction itself, but in any case, yes, given a bit of time and sufficient motivation, I don't think it's impenetrable either. So, while at it, let me share my most substantial worry about all this at the moment. While not impenetrable implies it's usable, I'm not sure how much further that goes. That's mostly fine if the only goal is to develop and run tests for passt (and I say "mostly" because to run these tests as part of automatic distribution testing you need to package them, and have packages for many distributions, which is a bit difficult to justify if you have a single usage, but let's set this aside for a moment). Still, that single-goal perspective doesn't look sustainable to me. That's the case for the current test suite, but it was never meant to be a real "framework" or simulator or anything anybody would like to use for anything else. If I'm looking for a tool that lets me quickly set up a VXLAN tunnel between two nodes and try to flip offloads on and off I think it's unreasonable to expect I'll go for some Scenario abstraction on the basis of being, after all... not impenetrable. And this kind of stuff is a very recurrent need in Linux networking development, in my experience, as well as an unsatisfied need in testing of many related projects. Of course, one pressing goal right now is to have a more structured way to define tests for passt, and anything that lets us achieve that goal with a reasonable amount of time and effort is welcome. But not having an interface that lets people build a test tunnel between two nodes in a couple of minutes of reading examples carries a serious risk that this gets stuck "forever" to passt and its tests. > > > + > > > + > > > +@UnconfiguredScenario.test > > > +@contextlib.contextmanager > > > +def simh_pasta_setup() -> Iterator[UnconfiguredScenario]: > > > + with (tunbridge.sample.simple_host('host') as simh, > > > + tunbridge.sample.isolated('guest', simh.site) as guest): > > > + assert simh.ip6 is not None > > > + assert simh.gw6_ll is not None > > > + with tasst.pasta.pasta(simh.site, guest): > > > + yield UnconfiguredScenario(host=3Dsimh.site, > > > + guest=3Dguest, > > > + ifname=3Dsimh.ifname, > > > + addr6=3Dsimh.ip6, > > > + gw6=3Dsimh.gw6_ll) =20 > >=20 > > ...and this too. > >=20 > > But there's one thing I'm missing: if it's a network simulator, why do > > you need to call a simple_host() method to *describe* the fact that you > > have a host / site? That looks rather unexpected. > >=20 > > I mean, I would have expected a syntax, in pseudocode, expressing: > >=20 > > 1. x :=3D node (properties such as a list of interfaces a, b, c) > >=20 > > 2. pasta implements/connects a > >=20 > > ...I think this is mostly embedded in the sample.simple_host() thing, > > but I'm not sure how. Maybe it will become clearer once I actually look > > into tunbridge, though. =20 >=20 > Right. "simple_host" isn't just an arbitrary node, but a (small) > predefined network topology: a node configured with a single default > gateway (also simulated, albeit minimally) - that is, the "classic" > pasta host. The idea is that the tunbridge.sample module will have a > bunch of such example networks - so far there's: > - isolated() (node with loopback only) > - back_to_back() (two nodes connected by a veth) > - simple_host() >=20 > Suggestions for better names welcome, as always. I'm a bit worried by the mere fact that those example networks (and they're all methods instead of some kind of grammar!) are needed. Anyway, I don't find back_to_back() particularly descriptive (what makes it not front-to-front?). Perhaps a more mundane "two_nodes()" makes it more obvious (they won't be isolated, of course). > > Of course, I'm trying to push away my bias coming from the fact I was, > > several years ago, for kselftests, aiming at something like this > > instead: > >=20 > > =09A veth B > > =09x=3D$(addr A veth) > > =09B ping -c1 $x > > =09A $x vxlan B $(addr B veth) > > =09... > >=20 > > (where 'veth', 'vxlan' were both reserved keywords). Maybe once > > non-trivial links are implemented in tunbridge it will all become more > > obvious. =20 >=20 > I think tunbridge is not dissimilar to this, though with functions > rather than reserved words. That's pretty much the whole difference I was trying to convey, though. Syntax is not entirely irrelevant. Of course, it doesn't need to be reserved words in arbitrary positions, but probably there are other ways to consider. > It's a bit hidden here, because we're > using these pre-built chunks - I expect that would be the case for > your system as well, once you get to complex enough setups that you > want to re-use non-trivial pieces. >=20 > For example the guts of back_to_back() is: >=20 > with isolated(f'{name}0', sb) as s0, \ > isolated(f'{name}1', sb) as s1: > if0, if1 =3D f'veth{name}0', f'veth{name}1' > with veth.veth(s0, if0, s1, if1): > =09 ... >=20 > There's more, but that's mostly about IP allocation (it optionally > does that). >=20 > > > + > > > + > > > +if __name__ =3D=3D '__main__': > > > + exeter.main() > > > diff --git a/test/run b/test/run > > > index 3872a56e..4f09d767 100755 > > > --- a/test/run > > > +++ b/test/run > > > @@ -43,8 +43,10 @@ KERNEL=3D${KERNEL:-"/boot/vmlinuz-$(uname -r)"} > > > =20 > > > COMMIT=3D"$(git log --oneline --no-decorate -1)" > > > =20 > > > -# Let exeter tests written in Python find their modules > > > +# Let exeter tests written in Python find their modules and binaries= to run > > > export PYTHONPATH=3D${BASEPATH}/exeter/py3:${BASEPATH}/tunbridge:${B= ASEPATH} > > > +export PASTA=3D${PASTA:-${BASEPATH}/../pasta} > > > + > > > =20 > > > . lib/util > > > . lib/context > > > @@ -75,8 +77,8 @@ run() { > > > =09exeter build/build.py > > > =09exeter build/static_checkers.sh > > > =20 > > > +=09exeter pasta/ndp.py > > > =09setup pasta > > > -=09test pasta/ndp > > > =09test pasta/dhcp > > > =09test pasta/tcp > > > =09test pasta/udp > > > diff --git a/test/tasst/__init__.py b/test/tasst/__init__.py > > > index fd4fe9a8..f5386b3a 100644 > > > --- a/test/tasst/__init__.py > > > +++ b/test/tasst/__init__.py > > > @@ -8,3 +8,7 @@ > > > # > > > # Copyright Red Hat > > > # Author: David Gibson > > > + > > > +from . import pasta > > > + > > > +__all__ =3D ['pasta'] > > > diff --git a/test/tasst/pasta.py b/test/tasst/pasta.py > > > new file mode 100644 > > > index 00000000..91f59036 > > > --- /dev/null > > > +++ b/test/tasst/pasta.py > > > @@ -0,0 +1,40 @@ > > > +#! /usr/bin/env python3 > > > +# > > > +# SPDX-License-Identifier: GPL-2.0-or-later > > > +# > > > +# TASST - Test A Simple Socket Transport > > > +# > > > +# test/tasst/pasta.py - Helpers for seeting up pasta instances > > > +# > > > +# Copyright Red Hat > > > +# Author: David Gibson > > > + > > > +import contextlib > > > +import os > > > +from typing import Iterator > > > + > > > +import tunbridge > > > + > > > + > > > +@contextlib.contextmanager > > > +def pasta(host: tunbridge.Site, guest: tunbridge.Site, *opts: str) \ > > > + -> Iterator[tunbridge.site.SiteProcess]: > > > + if tunbridge.unshare.parent(guest) is not host: > > > + raise ValueError("pasta guest must be a namespace under host= site") > > > + > > > + # This implies guest is a namespace site > > > + assert isinstance(guest, tunbridge.unshare.NsenterSite) > > > + > > > + exe =3D os.environ['PASTA'] > > > + > > > + with host.tempdir() as piddir: > > > + pidfile =3D os.path.join(piddir, 'pasta.pid') > > > + cmd =3D [exe, '-f', '-P', pidfile] + list(opts) + [f'{guest.= pid}'] > > > + with host.bg(*cmd, stop=3DTrue) as pasta: > > > + # Wait for the PID file to be written > > > + pidstr =3D None > > > + while not pidstr: > > > + pidstr =3D host.readfile(pidfile, check=3DFalse) > > > + pid =3D int(pidstr) > > > + print(f'pasta started, host: {host}, guest: {guest}, pid= : {pid}') > > > + yield pasta =20 > >=20 > > ...perhaps we could also print version and path. =20 >=20 > Path I can easily add. Version would require an extra invocation of > pasta, which I don't really want to do. Ah, right, never mind. The path will be good enough for that. > > This part also looks > > quite readable and intuitive to me without having looked into tunbridge > > recently. =20 >=20 > Ok, that's promising. I mean, I think it's all usable for the moment, and perhaps a starting point for some other kind of... front-end? I'm not sure. As I mentioned I'm a bit worried about the potential for universal intuitiveness and usability. --=20 Stefano