Re: Thoughts on interface modes / multiple guest addresses

[David Gibson <david@gibson.dropbear.id.au>]

[-- Attachment #1: Type: text/plain, Size: 8899 bytes --]

On Wed, Dec 17, 2025 at 03:01:31PM -0500, Jon Maloy wrote:
> 
> 
> On 2025-12-16 21:01, David Gibson wrote:
> > On Wed, Dec 17, 2025 at 01:29:36AM +0100, Stefano Brivio wrote:
> > > On Tue, 16 Dec 2025 16:53:49 +1100
> > > David Gibson <david@gibson.dropbear.id.au> wrote:
> > > 
> > > > Hi Jon,
> > > > 
> > > > As discussed on the call yesterday, I've written up my thoughts on
> > > > what a bunch of the address semantics should be.  Turns out I'd
> > > > already done some of this at:
> > > >      https://pad.passt.top/p/InterfaceMode
> > > 
> > > Two general comments:
> > > 
> > > 1. local mode is already implemented, and some things such as the
> > >     interface name ("tap0") are already defined, see man page and
> > >     'pasta -- pasta --config-net ip a'
> > 
> > Yes, I'm aware.  The two modes have the "normal" and "local" local to
> > indicate the existing modes they're more similar to (neither is
> > identical to current behaviour).  Another point I left out is that
> > this is intended as an endpoint to aim at.  Getting there I expect to
> > involve extra stuff for compatiblity along the way.
> > 
> > > 2. I think it's more relevant to define the basics of how one switches
> > >     between the existing local mode and a mode where we copy addresses
> > >     and routes (as they become available on the host), rather than
> > >     defining every single detail of these modes.
> > 
> > Oh.. right.  I guess I did't make this clear: these are modes set by
> > the command line (details TBD), we never switch between them at
> > runtime.  They kind of have to be, because which mode we're in affects
> > how we respond to runtime changes.
> > 
> > >     In these terms, I think it would actually be helpful to *avoid*
> > >     seeing them as separate modes. If there's no host connectivity, we'll
> > >     start in local mode, and switch to the other mode as we get addresses
> > >     and routes configured... just to switch back to the previous mode if
> > >     we lose them.
> > 
> > The whole point of all-interface mode (better name suggestions
> > welcome) is that the guest's routing configuration *doesn't* change,
> > even if the host's does.
> > 
> > Advantages:
> >   * The host can have whatever source-dependent, multi-path, bizarro
> >     routing configuration, changing as often as it likes and we (and
> >     the guest) don't care.  Guest routes to the host, host routes
> >     onward from there.
> 
> And the other way around, the guest should be able to set up whatever
> he wants if that is his choice, without affecting the host. Maybe
> even multiple interfaces, for whatever reason that might be.

I'm not sure I follow.  We can't change net config on the host, so
that's basically always true - in both proposed modes, and right now.
So maybe you're meaning something different?  I'm not sure what,
though.

> We could have two modes: "transparent" and "opaque"

I'm not 100% sold, but I think those are at least a bit better than my
current names (doc updated).

> In opaque mode we get basically what you describe above, plus that we allow
> the guest to add new addresses/routes in runtime.

THe guest could always add extra routes at runtime.  Adding extra
addresses... well, it always could, the question is what - if anything
- passt/pasta should do about it.  What do you propose?

> So, we keep the multi-address configuration and and the nameaspace side
> subscription, but block host-side subscriptions in this mode.

My intention was that we don't use a host-side *route* subscription in
this mode.  We would still need a host-side address subscription to
implement '-a auto', which I was proposing to have allowed (and
enabled by default) in opaque mode.

> Conversely, we are fully open to host-changes in transparent mode, we
> subscribe for host-side changes, forward those to the namespace,

With --config-net, yes.  For passt and !--config-net we can't control
the guest configuration; we can only reflect as much of the host state
as possible via DHCP and NDP, and hope the guest will consult that and
update itself.

> However, we don't allow the guest user to manipulate anything in run-time.

Well, we can't really prevent it.  I guess you mean if the guest
reconfigures itself, then things might break entirely and it's not our
fault?

> Hybrid modes might be possible,

I haven't (so far) seen a sensible way to hybridise these models.  The
main sticking point is giving a consistent meaning to link-local
addresses when used by the guest.

> e.g., that we allow one stable link local
> address even in transparent mode.

That's allowed by the proposed model: transparent mode with '-a <ll
addr> -a auto'.

> Would that make sense?
> 
> >   * We have a consistent (link local) way of addressing the host
> >     regardless of what changes happen on the host side
> >   * We have the freedom to allocate link-local addresses if we want
> >     them for any purpose
> > Disadvantages:
> >   * No access to external peers via link-local address
> >   * Guest's routing setup is visibly different from the host's (so less
> >     L3 transparent)
> > 
> > I actually think that's a more useful and robust way of operating for
> > most use cases and we should eventually make it the default.
> > One-interface mode is for the use cases where those disadvantages are
> > fatal.
> > 
> > There is another possible option here: present multiple interfaces in
> > the guest, one for each host interface.
> 
> In transparent mode that would be a natural further step.

Yes.  Supporting multiple guest side pifs is a preqrequisite, though.
I'm working towards that, but it's a fair way off.

> 
> ///jon
> 
> > I'm not including it, since
> > it's basically equivalent to having multiple pasta instances in
> > one-interface mode.  To implement this, we'd basically have to
> > implement one-interface mode first.
> > 
> > >     So does it really help to have "modes" instead of just considering
> > >     what addresses and routes are we going to delete, and when? Because
> > >     that's what we'll need to do anyway (and that's what I think defines
> > >     the design).
> > 
> > I haven't seen a way to define coherent semantics that cover all the
> > use cases without introducing two modes.  The overlapping constraints
> > here are:
> > 
> >   * With passt or !--config-net, we can't fully control the guest's
> >     networking config.  We both can't set things with arbitrary
> >     precision, and we don't have a way of forcing an update when things
> >     change.
> >   * If we're dealing with multiple host interfaces - either
> >     concurrently or to a lesser extent over time, then there's no way
> >     to coherently map host-side link-local addresses to the guest.
> > 
> > >     I see that this is not an explicit use case in Jon's list (which I
> > >     still have to review), but it's one of the most two fundamental ones
> > >     I think (that, and Podman Quadlets), also nicely described by a user
> > >     at:
> > > 
> > >       https://github.com/containers/podman/discussions/22737#discussioncomment-9478727
> > 
> > Ah, yes, that is another case.   I think it would work out equivalent
> > to one-interface mode attached to a dummy0 interface on the host.  So,
> > it should be fairly easy to implement in terms of one-interface mode,
> > just pretending a dummy0 existed even if it doesn't.
> > 
> > > > I've now updated to cover some more things, and considering the
> > > > possibility of multiple guest addresses..  Turns out etherpad doesn't
> > > > really do tables, so it's two sections for the two suggested modes,
> > > > with matching subheadings.
> > > 
> > > It does, but I disabled the plug-in as you reported an issue which
> > > turned out to be https://github.com/bitwarden/clients/issues/17598
> > > instead, and I was trying to sort out other possible reasons.
> > > 
> > > I just re-enabled it, tables are available from the toolbar, there's
> > > an icon just left of "Font Family". Note that it's still beta:
> > > 
> > >    https://www.npmjs.com/package/ep_data_tables
> > > 
> > > and it has a couple of glitches. I just found one (which I didn't debug
> > > or report yet): don't start a page with a table, always write something
> > > before, otherwise it gets duplicated every time you load the document.
> > > 
> > > Other than that it looks reasonably robust to me, maybe quickly try with
> > > a test pad first but I think it should be usable.
> > 
> > Great, thanks.
> > 
> 

-- 
David Gibson (he or they)	| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you, not the other way
				| around.
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]