Re: [PATCH 3/3] test: Re-implement pasta NDP tests using tunbridge & exeter

[David Gibson <david@gibson.dropbear.id.au>]

[-- Attachment #1: Type: text/plain, Size: 31537 bytes --]

On Wed, Oct 29, 2025 at 12:17:22AM +0100, Stefano Brivio wrote:
> On Fri, 17 Oct 2025 15:30:33 +1100
> David Gibson <david@gibson.dropbear.id.au> wrote:
> > On Thu, Oct 16, 2025 at 11:31:19PM +0200, Stefano Brivio wrote:
> > > On Fri, 10 Oct 2025 13:17:13 +1100
> > > David Gibson <david@gibson.dropbear.id.au> wrote:
> > >   
> > > > On Fri, Oct 10, 2025 at 01:20:23AM +0200, Stefano Brivio wrote:  
> > > > > On Thu, 9 Oct 2025 15:47:01 +1100
> > > > > David Gibson <david@gibson.dropbear.id.au> wrote:
> > > > >     
> > > > > > On Thu, Oct 09, 2025 at 01:02:48AM +0200, Stefano Brivio wrote:    
> > > > > > > On Wed, 8 Oct 2025 13:32:27 +1100
> > > > > > > David Gibson <david@gibson.dropbear.id.au> wrote:
> > > > > > >       
> > > > > > > > On Tue, Oct 07, 2025 at 10:01:10PM +0200, Stefano Brivio wrote:      
> > > > > > > > > On Thu,  2 Oct 2025 17:57:08 +1000
> > > > > > > > > David Gibson <david@gibson.dropbear.id.au> wrote:
> > > > > > > > >         
> > > > > > > > > > Convert the pasta NDP tests from shell and our own DSL to Python using
> > > > > > > > > > the exeter test protocol and tunbridge network simulation library.
> > > > > > > > > > 
> > > > > > > > > > Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> > > > > > > > > > ---
> > > > > > > > > >  test/Makefile          |  2 +-
> > > > > > > > > >  test/pasta/dhcp        |  5 ++++
> > > > > > > > > >  test/pasta/ndp.py      | 59 ++++++++++++++++++++++++++++++++++++++++++
> > > > > > > > > >  test/run               |  6 +++--
> > > > > > > > > >  test/tasst/__init__.py |  4 +++
> > > > > > > > > >  test/tasst/pasta.py    | 40 ++++++++++++++++++++++++++++
> > > > > > > > > >  6 files changed, 113 insertions(+), 3 deletions(-)
> > > > > > > > > >  create mode 100755 test/pasta/ndp.py
> > > > > > > > > >  create mode 100644 test/tasst/pasta.py
> > > > > > > > > > 
> > > > > > > > > > diff --git a/test/Makefile b/test/Makefile
> > > > > > > > > > index f66c7e7e..95e3d75e 100644
> > > > > > > > > > --- a/test/Makefile
> > > > > > > > > > +++ b/test/Makefile
> > > > > > > > > > @@ -67,7 +67,7 @@ ASSETS = $(DOWNLOAD_ASSETS) $(LOCAL_ASSETS)
> > > > > > > > > >  
> > > > > > > > > >  EXETER_SH = smoke/smoke.sh build/static_checkers.sh
> > > > > > > > > >  EXETER_PYPATH = exeter/py3:tunbridge/:.
> > > > > > > > > > -EXETER_PYTHON = smoke/smoke.py build/build.py
> > > > > > > > > > +EXETER_PYTHON = smoke/smoke.py build/build.py pasta/ndp.py
> > > > > > > > > >  EXETER_BATS = $(EXETER_SH:%=%.bats) $(EXETER_PYTHON:%=%.bats)
> > > > > > > > > >  BATS_FILES = $(EXETER_BATS) \
> > > > > > > > > >  	podman/test/system/505-networking-pasta.bats
> > > > > > > > > > diff --git a/test/pasta/dhcp b/test/pasta/dhcp
> > > > > > > > > > index e1c66be6..61279fbf 100644
> > > > > > > > > > --- a/test/pasta/dhcp
> > > > > > > > > > +++ b/test/pasta/dhcp
> > > > > > > > > > @@ -18,6 +18,11 @@ test	Interface name
> > > > > > > > > >  nsout	IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname'
> > > > > > > > > >  check	[ -n "__IFNAME__" ]
> > > > > > > > > >  
> > > > > > > > > > +# Bring up the interface
> > > > > > > > > > +ns	ip link set dev __IFNAME__ up
> > > > > > > > > > +# Wait for SLAAC & DAD to complete
> > > > > > > > > > +ns	while ! ip -j -6 addr show dev __IFNAME__ | jq -e '.[].addr_info.[] | select(.protocol == "kernel_ra")'; do sleep 0.1; done
> > > > > > > > > > +
> > > > > > > > > >  test	DHCP: address
> > > > > > > > > >  ns	/sbin/dhclient -4 --no-pid __IFNAME__
> > > > > > > > > >  nsout	ADDR ip -j -4 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[0].local'
> > > > > > > > > > diff --git a/test/pasta/ndp.py b/test/pasta/ndp.py
> > > > > > > > > > new file mode 100755
> > > > > > > > > > index 00000000..8c7ce31e
> > > > > > > > > > --- /dev/null
> > > > > > > > > > +++ b/test/pasta/ndp.py
> > > > > > > > > > @@ -0,0 +1,59 @@
> > > > > > > > > > +#! /usr/bin/env python3
> > > > > > > > > > +#
> > > > > > > > > > +# SPDX-License-Identifier: GPL-2.0-or-later
> > > > > > > > > > +#
> > > > > > > > > > +# test/pasta/ndp.py - pasta NDP functionality
> > > > > > > > > > +#
> > > > > > > > > > +# Copyright Red Hat
> > > > > > > > > > +# Author: David Gibson <david@gibson.dropbear.id.au>
> > > > > > > > > > +
> > > > > > > > > > +import contextlib
> > > > > > > > > > +import dataclasses
> > > > > > > > > > +from typing import Iterator
> > > > > > > > > > +
> > > > > > > > > > +import exeter
> > > > > > > > > > +import tunbridge
> > > > > > > > > > +import tasst
> > > > > > > > > > +
> > > > > > > > > > +
> > > > > > > > > > +@dataclasses.dataclass
> > > > > > > > > > +class UnconfiguredScenario(exeter.Scenario):
> > > > > > > > > > +    """Tests for a pasta instance without --config-net"""
> > > > > > > > > > +
> > > > > > > > > > +    host: tunbridge.Site
> > > > > > > > > > +    guest: tunbridge.Site
> > > > > > > > > > +    ifname: str
> > > > > > > > > > +    addr6: tunbridge.ip.AddrMask6
> > > > > > > > > > +    gw6: tunbridge.ip.Addr6        
> > > > > > > > > 
> > > > > > > > > Until this point, it looks like stuff I can happily copy and paste,
> > > > > > > > > and grasp, even. But then:
> > > > > > > > >         
> > > > > > > > > > +    @exeter.scenariotest
> > > > > > > > > > +    def test_ifname(self) -> None:
> > > > > > > > > > +        ifs = tunbridge.ip.ifs(self.guest)
> > > > > > > > > > +        exeter.assert_eq(set(ifs), {'lo', self.ifname})        
> > > > > > > > > 
> > > > > > > > > ...why does a "Scenario" have a .ifname?        
> > > > > > > > 
> > > > > > > > Yeah, the readability of the Scenario mechanism was something I was
> > > > > > > > particularly concerned about.  I think the concept is valuable, but
> > > > > > > > I'm very open to different ways of naming or organising it, if we can
> > > > > > > > up with something better.      
> > > > > > > 
> > > > > > > From the description you give below, the name seems to fit.
> > > > > > >       
> > > > > > > > A "Scenario" (specifically a subclass of exeter.Scenario) is a group
> > > > > > > > of tests with a common set of parameters.  In this case
> > > > > > > > UnconfiguredScenario is a bunch of tests about the behaviour of pasta
> > > > > > > > without --config-net.  Each of those tests has access to the host and
> > > > > > > > guest sites, the expected interface name, address and gateway in the
> > > > > > > > guest - that is, the contents of an UncofiguredScenario instance.      
> > > > > > > 
> > > > > > > I'm not sure if I understand this correctly, but if each guest has a
> > > > > > > single interface, that sounds a bit limiting.      
> > > > > > 
> > > > > > Sorry, to be clear: a Scenario in the general sense can contain
> > > > > > whatever parameters you like.  This *particular* Scenario -
> > > > > > UnconfiguredScenario - has just those things, because those are all
> > > > > > that its tests require.    
> > > > > 
> > > > > Ah, okay. Still, if I now want to take UnconfiguredScenario and add a
> > > > > couple of dummy interfaces to it for a quick test, I guess I have the
> > > > > choice to either do that with some "external" hack, or... copy and
> > > > > rename it, so that it doesn't affect all the usages?    
> > > > 
> > > > No.  A Scenario instance isn't responsible for managing the simulated
> > > > environment - that's the setup function - it's just conveying the
> > > > information about it that the tests need.  So, you can make a setup
> > > > function that adds the dummy interfaces, and still yield an
> > > > UnconfiguredScenario.  It doesn't need to have information about the
> > > > dummy interfaces because the tests carried by UnconfiguredScenario
> > > > don't care about them.  
> > > 
> > > Oh, sorry, it's a class, of course, I see now.
> > >   
> > > > The scenario mechanism does several things:
> > > >   1) Groups together some related (parameterized) tests
> > > >   2) Allows all of those tests to be registered at once
> > > >   3) Provides a mechanism for providing a bunch of information to
> > > >      those tests (without requiring them each to have a large set of
> > > >      direct parameters)
> > > > 
> > > > I'm aware that doing those things with the same construct may be
> > > > confusing - it's just ways of doing them separately also seem
> > > > confusing and/or awkward in their own ways.  Maybe there's a better
> > > > way, but I haven't spotted it yet.  
> > > 
> > > It really is confusing to me, but the description above is rather clear
> > > so I'll try to propose something once I get to write some kind of setup
> > > function and test cases myself.  
> > 
> > Makes sense.
> > 
> > > > > > > Actually, I think any abstraction that doesn't offer arbitrary sets of
> > > > > > > (and relationships between) the objects shown via netlink (or, at
> > > > > > > least, namespaces, links, routes, addresses, neighbours) might be
> > > > > > > limiting and not generic enough.      
> > > > > > 
> > > > > > Absolutely, and the abstraction does allow that.
> > > > > >     
> > > > > > > > That instance describes a real (simulated) environment in which we can
> > > > > > > > run those tests.
> > > > > > > > 
> > > > > > > > You use this by supplying a function which sets things up, then yields
> > > > > > > > an UnconfiguredScenario instance describing what it set up.  exeter
> > > > > > > > will run all of the UnconfiguredScenario tests on the environment the
> > > > > > > > setup function created, each one as a separate test case.      
> > > > > > > 
> > > > > > > This part is now clear to me, and I think it's not complicated to grasp
> > > > > > > the concept vaguely but enough to copy, paste, and modify code doing
> > > > > > > this.      
> > > > > > 
> > > > > > Ok.
> > > > > >     
> > > > > > > It would be even better to hide this entirely, because "yielding a
> > > > > > > scenario" is a Python thing. In general, there's an imperative part in
> > > > > > > all this (bordering functional programming, but still, not descriptive)
> > > > > > > which I struggle to see as beneficial.
> > > > > > > 
> > > > > > > Here the tasks at hand are, roughly:
> > > > > > > 
> > > > > > > 1. represent two network namespaces, with two interfaces each (loopback
> > > > > > >    and non-loopback), with pasta connecting one of the interfaces of the
> > > > > > >    inner one      
> > > > > > 
> > > > > > There's a bit more to it than that - we need to specify the host's
> > > > > > routing setup, because that will affect what pasta does.  That's what
> > > > > > simple_host() is about, creating a host with the single gateway
> > > > > > routing that's our easiest / most common case.    
> > > > > 
> > > > > Okay, sure, by "interfaces" I meant configured interfaces with
> > > > > addresses and a default route, too. But that doesn't really modify my
> > > > > point, that is:
> > > > >     
> > > > > > > 2. bring up one of the interfaces
> > > > > > > 
> > > > > > > 3. compare addresses
> > > > > > > 
> > > > > > > ...and doing 1. like that is simply not... intuitive, I think.      
> > > > > > 
> > > > > > I'm not really clear on what you're getting at here.  There is an
> > > > > > unavoidable tradeoff here between obviousness for a single case,
> > > > > > versus reuseability for multiple related cases.  Is it just that some
> > > > > > of the relevant setup is hidden inside simple_host() that's the
> > > > > > problem? Or is it something else?    
> > > > > 
> > > > > ...yes, one part is that it's hidden. Another part are, specifically,
> > > > > these lines:
> > > > > 
> > > > > 	host: tunbridge.Site
> > > > > 	guest: tunbridge.Site
> > > > > 	ifname: str
> > > > > 
> > > > > 	[...]
> > > > > 
> > > > > 	@exeter.scenariotest
> > > > > 	def test_ifname(self) -> None:
> > > > > 
> > > > > 	[...]
> > > > > 
> > > > > None of these clearly links to "two network namespaces: A, with
> > > > > interface a1 and address x1, ...".    
> > > > 
> > > > Fair.  This needs a docstring explaining the parameters / fields.  
> > > 
> > > That might help a tiny bit but I think the syntax and notations are
> > > kind of self-explanatory.
> > > 
> > > My concern is at a more conceptual level, and it's better summarised
> > > below, but here, specifically, we're writing:
> > > 
> > >   host: tunbridge.Site
> > > 
> > > to say:
> > > 
> > >   give me the "host" network namespace  
> > 
> > What's "me" in this sentence?
> 
> The writer of the test case, that is:

So, I'm pretty sure we have a fundamental misunderstanding here, but
I'm not sure on whose part.

That line is not saying "give me" anything.  It's a type declaration
saying that an 'UnconfiguredScenario' instance includes a field 'host'
of type tunbridge.Site.  That field is used to store the host
namespace; the docstring should say that, but doesn't yet.

The tests (methods of UnconfiguredScenario) can then reference the
host namespace as 'self.host'.  The setup function is responsible for
populating the field - that's the 'host=simh.site' in
simh_pasta_setup() below.

> > > and to say that, in my ideal world, I would probably go for something
> > > on the line(s) of:
> > > 
> > >   A  
> > 
> > Not really following, I hope it will make more sense to me below.
> 
> if "host" is a site in "host: tunbridge.Site", which I'm calling "A"
> for brevity, I would like to say that with:
> 
> 	A
> 
> instead of:
> 
> 	A: tunbridge.Site

In the test?  So it's already 'self.host', or 'self.A' if you prefer.
You only need the type annotation when defining a new Scenario
subclass.  Or we could disable mypy and not even need it then, but I
think type checking catches enough bugs to make it worthwhile.

> > > > > I understand this is probably very close to the bare minimum you can
> > > > > get by modelling this all with actual code, and that's why I think
> > > > > actual (imperative/functional) code is usually not used to
> > > > > model/describe things.    
> > > > 
> > > > Imperative/functional code as opposed to..?  
> > > 
> > > ...declarative.  
> > 
> > Ok, I've generally seen functional programming put under the
> > declarative heading.  Or as my exam invigilator famously said once
> > "Declaraytive Programming Paradijjums".
> > 
> > So, I guess that makes me unclear what does and doesn't count as
> > declarative for you.
> 
> Hmm yes I see your point. I would have said that "purely functional
> programming" is declarative, and "functional programming" is a mix of
> declarative and imperative (in practice).

Eh, true.  Kind of depends if you're talking Haskell or, say, Emacs
Lisp.  And even in Haskell 'do' constructs are quasi-imperative.

> In any case, I guess I later clarified this aspect.
> 
> > [snip]
> > > > > > Syntax certainly isn't irrelevant, but so far I haven't grasped what
> > > > > > you dislike about the function syntax versus a specialized grammer.
> > > > > > Is it:
> > > > > >   - The irritating silly parentheses?
> > > > > >   - Longish (qualified) function names?
> > > > > >   - The indentation from the with syntax?
> > > > > >   - Something else?    
> > > > > 
> > > > > It's *also* the first two    
> > > > 
> > > > Ok.  First I can't easily change.  Second can be mitigated by handling
> > > > the imports differently.
> > > >   
> > > > > (the indentation looks actually convenient),    
> > > > 
> > > > Ok, good.  I also think this is useful because it conveys the lifetime
> > > > of each object, which will be important once we get to tests where you
> > > > need to change things part way through.
> > > >   
> > > > > but that's not my main point. My main point is that this isn't
> > > > > fundamentally declarative. You're turning it into something that
> > > > > resembles that, but the syntax is still from an imperative programming
> > > > > language.
> > > > > 
> > > > > And in my mind the main feature of a network (topology) simulator is
> > > > > that you describe the topology (and it will build it for you), not that
> > > > > you... have to build a description?
> > > > > 
> > > > > Using an example that's obviously familiar to you: think of taking a
> > > > > device tree for some system with a couple of USB and I²C busses and a
> > > > > flash controller, and writing all that in Python based on some form of
> > > > > "bus" module/component.    
> > > > 
> > > > I mean... old school Open Firmware kind of is this, but with Forth
> > > > instead of Python.  
> > > 
> > > Okay, you can model data structures in Python, obviously, but that
> > > wasn't my point. Anyway, it's all in the example below.
> > >   
> > > > > Once one sees how practical device trees are for that, the Python
> > > > > version would look wrong, wouldn't it?    
> > > > 
> > > > That really depends on the context.  If I was making an API for
> > > > building a device tree, I'd probably come up with something pretty
> > > > like this.  
> > > 
> > > ...an API for building one, yes. But not if you were writing a device
> > > tree itself.
> > >   
> > > > > Now, while I think that some single bits of DTS syntax are
> > > > > unnecessarily complicated, conceptually, a "networking" device tree
> > > > > would look more usable to me than the approach you're taking.
> > > > > 
> > > > > Of course, we need the whole "testing" / exeter part as well, and test
> > > > > cases are fundamentally sequential/imperative.
> > > > > 
> > > > > But (sorry, it's been a few years I don't touch these):
> > > > > 
> > > > > namespace@1 {
> > > > >   interfaces {
> > > > >     lo {
> > > > >       address = 127.0.0.1;
> > > > >     };
> > > > >     eth0 {
> > > > >       address = ...;
> > > > >     };
> > > > >   };
> > > > >   routes {
> > > > >     /* something simpler than ip -j ro sh ? */
> > > > >   };
> > > > > }
> > > > > 
> > > > > ...
> > > > > 
> > > > > link@... {
> > > > >   vxlan {
> > > > >     endpoints {
> > > > >       a {
> > > > >          ns = <&namespace@1>;
> > > > >       };
> > > > >       b ...
> > > > > 
> > > > > ...
> > > > >       
> > > > > this looks much more natural to me, as an input for a simulator (I
> > > > > would personally make the syntax much more "elastic" by just throwing a
> > > > > link into a namespace but I'm trying to keep it clean just for this
> > > > > example).    
> > > > 
> > > > Aha, I think I finally get what you're saying.  More below.
> > > >   
> > > > > Maybe tunbridge implements this somewhere and I missed it? Or would
> > > > > this be part of a "Scenario" description eventually?    
> > > > 
> > > > This is entirely unrelated to what Scenario is trying to accomplish.
> > > > That may cause you to reconsider whether "Scenario" is a good name,
> > > > which is ok.
> > > > 
> > > > 
> > > > So.  A declarative way of defining networks would be nice to have.  
> > > 
> > > From my perspective that's fundamental, rather. I gave it for granted.
> > >   
> > > > I think doing it with the flexibility we want is much harder than you
> > > > estimate.  
> > > 
> > > I'll pretend I'm not commenting on this line by... oops. :)  
> > 
> > Heh.  Well, I'd love to be proved wrong on this one.
> > 
> > > > It looks easy for simple static situations like the
> > > > examples above, but:
> > > > 
> > > >  * If you want to describe a topology that changes partway through,
> > > >    that's a huge step up in complexity, and kind of necessarily
> > > >    reintroduces imperative elements.  
> > > 
> > > But you can use JSON or a ton of other formats that allow for ordering
> > > of elements.  
> > 
> > Just ordering network elements w.r.t. each other isn't enough.  If you
> > have a test where you want /this/ network topology - do some stuff -
> > then change the toplogy to /that/ - do some more stuff.  I'm not sure
> > how you encode that in a way that isn't imperative, or at least
> > pseudo-imperative.
> 
> Yeah, I'll come up with a proposal, this is rather clear to me: simply
> interleave declarative setups with imperative test steps.
> 
> > > Alternatively, one could add attributes for lifecycle and
> > > timing (think of nftables sets) but it looks much less intuitive than
> > > the alternative.  
> > 
> > I wasn't previously familiar with nftables sets.  I had a quick look
> > at the docs, but I don't see how it relates to the current proposal.
> 
> Sorry, I took this for granted: I meant timeout attributes for, say,
> elements in sets. That is, making the lifecycle validity an attribute
> of some declared object, rather than encapsulating declarative blocks
> in a possibly imperative sequence (if needed).
> 
> > > >    Device tree absolutely suffers
> > > >    from this - that's what motivated the godawful runtime overlay
> > > >    mechanism, and right now, I'm struggling to find time to
> > > >    participate in the latest of many discussions about how to better
> > > >    handle devices which can be runtime added and removed.  
> > > 
> > > I'm not suggesting that we use ANS Forth by the way.  
> > 
> > Well, we agree on that at least :).
> > 
> > > >  * If you want to build complex scenarios out of simpler ones, you
> > > >    need what amounts to a macro system.  
> > > 
> > > There are a ton of ways. You can also use a filesystem and includes. Or
> > > simply definitions of blocks, not necessarily macros,  
> > 
> > You want to parameterise the blocks - at which point it's basically
> > macros.
> > 
> > > and JSON
> > > implicitly gives you all that.  
> > 
> > No, it doesn't.  If you have a JSON sub-object that's repeated several
> > times through a tree, you have to write it out, in full, each time.
> > YAML does allow repeating, as do some other JSON extensions, but bare
> > JSON does not.  It still doesn't allow parameterisation of those
> > blocks.
> 
> Until you meet RFC 6901.

Only sort of.  That's describing a convention for how to reference
JSON nodes with a string.  It's still not a notion in JSON proper -
whatever's consuming the JSON has to know to expect an RFC 6901
pointer and process it accordingly.  It doesn't help in the slightest
with parameterization.

> > > As it's nothing security relevant, I would actually go with something
> > > that's in theory more complicated but in practice more digestible such
> > > as YAML.  
> > 
> > I don't terribly like YAML, because I think there are a bunch of edge
> > cases where it's not obvious reading it whether something is a list or
> > object or somethine else.  The typing and back-referencing would be
> > useful for this problem, I'll grant you.
> 
> ...so perhaps YAML as an optional human-barely-tolerating format that we
> translate to JSON?

Maybe.

> Or TOML?

Maybe.  I like TOML in general, but I feel like it's specific syntax
isn't great for this use case.

> > > >    Again, a big leap up in
> > > >    complexity.  Device tree struggles with this too - it originated
> > > >    primarily as a machine->machine format, where having heaps of
> > > >    repeated information is fine.  As it transitioned to being a
> > > >    human->machine format, not so much.  Hence /include/, expression
> > > >    support and semi-standard running dts files through cpp before
> > > >    compilation.  It's still pretty clunky in this regard.  
> > > 
> > > It absolutely is, but that's because it was designed for a different
> > > purpose.
> > >   
> > > > Plus.. I think the interpreter for this hypothetical declarative
> > > > language would need an internal structure pretty similar to what
> > > > tunbridge, so this is kind of already a first step towards it.  
> > > 
> > > Okay, that's good to know.
> > > 
> > > I'm estimating I'm currently writing about 5-10 scripts per month,
> > > including pasta/iproute2 one-liners, setting up strange stuff, to
> > > reproduce / debug issues.
> > > 
> > > Given that this looks so fundamental for my usage I'm thinking that I
> > > could make at least part of this a priority of mine.
> > > 
> > > I realised I can implement netlink stuff and handling of networking
> > > configuration concepts quite fast with Rust and neli, so I'm pondering
> > > to write a proof of concept that can parse the example above (minus
> > > Forth notations, but with some kind of pointer) and make it create at
> > > least namespaces, links, addresses, and routes.
> > > 
> > > If it helps visualising how that could possibly look like with / in
> > > tunbridge itself, I'll take care of it soon rather than later.
> > > 
> > > The only little problem is that I'm much faster with Rust (because of
> > > neli) than I can possibly picture myself with Python, and that doesn't
> > > play along with tunbridge. But perhaps as a proof of concept it helps
> > > anyway?  
> > 
> > It would.  At earlier points I did consider writing tunbridge (or
> > whatever I was calling the idea at the time) in Rust.  In principle at
> > least, the lifetime tracking would make a very natural way for
> > ensuring you bring up / pull down things in a sensible order.
> > 
> > In practice, however, I almost instantly ran into intractable borrow
> > checker problems.  Specifically, Rust's notorious difficulty with
> > references between parts of the same structure.  That arises almost
> > immediately once you start building composite objects out of smaller
> > components:
> > 
> > 	struct Node { ... }
> > 	struct Veth<'a> { node0: &'a Node, node1: &'a Node, ... }
> > 
> > Then you want something that represents two nodes with a veth between
> > them and you get:
> > 
> > 	struct TwoNodes { node0: Node, node1: Node, veth: Veth<'??> }
> > 
> > There are ways around it, of course, but everything I found so far was
> > either hideously unergonomic, depended on obscure crates or both.
> > There are a bunch of language extensions that would help, but while
> > much discussed, none have landed yet.
> > 
> > If a way to solve this nicely appears, I'm not against moving
> > tunbridge to Rust.  I mean, slightly more against it than I was months
> > ago before I made a start in Python, but still.
> 
> Well, let's see how my draft turns out. I think there are obvious
> marketing reasons for Rust over Python, and a couple of technical ones
> too (speed, plus what I mentioned about neli), but there are also
> technical downsides as you point out.

Oh, agreed.  At the moment those technical downsides seem pretty fatal
to me, though.  Oh I missed an option there: hideously unergonomic,
depend on obscure crates or neuter the borrow checker even in the
cases it would be really useful.  Or several of the above.

> > > In general, do you think there's something in particular I could
> > > contribute at this stage, if I want to see my declarative dream come
> > > true?  
> > 
> > Yes.  A bunch of pseudo-code examples - both the network declarations
> > and example tests that might go with them.
> > 
> > [snip]
> > > > > > > > Path I can easily add.  Version would require an extra invocation of
> > > > > > > > pasta, which I don't really want to do.      
> > > > > > > 
> > > > > > > Ah, right, never mind. The path will be good enough for that.
> > > > > > >       
> > > > > > > > > This part also looks
> > > > > > > > > quite readable and intuitive to me without having looked into tunbridge
> > > > > > > > > recently.        
> > > > > > > > 
> > > > > > > > Ok, that's promising.      
> > > > > > > 
> > > > > > > I mean, I think it's all usable for the moment, and perhaps a starting
> > > > > > > point for some other kind of... front-end? I'm not sure. As I mentioned
> > > > > > > I'm a bit worried about the potential for universal intuitiveness and
> > > > > > > usability.      
> > > > > > 
> > > > > > So am I, but I have to weigh it against being able to re-use both
> > > > > > tests and setups without having to re-express both in each case.    
> > > > > 
> > > > > I think setups written like that are reusable (or can be made
> > > > > reusable). My usability point is about other project/usages. For passt
> > > > > and pasta themselves, this level or reusability looks enough to me for
> > > > > the foreseeable future.
> > > > > 
> > > > > Even though, one day, I guess we might want to generate pseudo-random
> > > > > (fractal-tree-like?) topologies (and I was recently trying out a
> > > > > pasta-in-pasta-in-pasta-in-pasta-in-pasta setup to reproduce that
> > > > > HTTP/FIN issue). For that, a declarative approach would make things
> > > > > easier, I suppose.    
> > > > 
> > > > Declarative, or imperative?  I actually have something like that in
> > > > tunbridge's selftests: a function that builds a stack of N nested
> > > > namespaces.
> > > >     https://gitlab.com/dgibson/tunbridge/-/blob/main/tunbridge/unshare.py#L302  
> > > 
> > > ...but they are all the same. Think, for example, of connecting every
> > > odd-numbered pair with veth tunnels, and every even-numbered pair with
> > > pasta. Say: n1 <-- veth --> n2 <-- pasta --> n3 <-- veth --> n4.  
> > 
> > That would certainly be possible.  More complex, of course, but not
> > dramtically so.
> > 
> > > What's really well suited for this situation, in my experience, is a
> > > declarative description format that can be easily generated and
> > > manipulated by imperative code.  
> > 
> > Ah, so there's both an imperative and declarative component.  The idea
> > in tunbridge is that you can do this, but rather than emit the
> > "declarative part" as concrete text for another parser, it's emitted
> > as a data structure (generally a bunch of wrappers around context
> > managers).
> > 
> > It is true that as currently designed, tunbridge builds the data
> > structure representation at the same time as building the actual
> > simulated network.  With a declarative language approach, building the
> > description (language fragment) is separate from instantiating the
> > simulation.  Is that something you see as valuable?  Or only a side
> > effect of the other things about the declarative approach you like?
> 
> I see that as valuable by itself, mostly because those fragments can be
> generated much more easily if they're separated from the imperative
> part.

Ok.  I'll look into separating 'construct the describing data
structure' from 'instantiating the simulation' in tunbridge.  It's not
everything you want, but it's a start (and would be a step closer to
something that could implement a more declarative approach).  Might
not be for the next spin though, it's a pretty substantial rework.

> > > The name of this kind of "indirection" in computer science research
> > > currently escapes me, but I'm fairly sure there must be some theory
> > > about it. In any case, I can include something like this in my (now
> > > planned) proof of concept.
> 
> ...still planned...

-- 
David Gibson (he or they)	| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you, not the other way
				| around.
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]