Thoughts on interface modes / multiple guest addresses

Thoughts on interface modes / multiple guest addresses

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 936 bytes --]

Hi Jon,

As discussed on the call yesterday, I've written up my thoughts on
what a bunch of the address semantics should be.  Turns out I'd
already done some of this at:
    https://pad.passt.top/p/InterfaceMode

I've now updated to cover some more things, and considering the
possibility of multiple guest addresses..  Turns out etherpad doesn't
really do tables, so it's two sections for the two suggested modes,
with matching subheadings.

It's still very much a draft so:
 * It might be unclear - please ask me to clarify anything
 * It will have some mistakes - make your case if you think
   my proposed semantics don't make sense
 * It will be incomplete - feel free to add extra subheadings for
   anything you 

-- 
David Gibson (he or they)	| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you, not the other way
				| around.
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Thoughts on interface modes / multiple guest addresses

[Stefano Brivio]

On Tue, 16 Dec 2025 16:53:49 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> Hi Jon,
> 
> As discussed on the call yesterday, I've written up my thoughts on
> what a bunch of the address semantics should be.  Turns out I'd
> already done some of this at:
>     https://pad.passt.top/p/InterfaceMode

Two general comments:

1. local mode is already implemented, and some things such as the
   interface name ("tap0") are already defined, see man page and
   'pasta -- pasta --config-net ip a'

2. I think it's more relevant to define the basics of how one switches
   between the existing local mode and a mode where we copy addresses
   and routes (as they become available on the host), rather than
   defining every single detail of these modes.

   In these terms, I think it would actually be helpful to *avoid*
   seeing them as separate modes. If there's no host connectivity, we'll
   start in local mode, and switch to the other mode as we get addresses
   and routes configured... just to switch back to the previous mode if
   we lose them.

   So does it really help to have "modes" instead of just considering
   what addresses and routes are we going to delete, and when? Because
   that's what we'll need to do anyway (and that's what I think defines
   the design).

   I see that this is not an explicit use case in Jon's list (which I
   still have to review), but it's one of the most two fundamental ones
   I think (that, and Podman Quadlets), also nicely described by a user
   at:

     https://github.com/containers/podman/discussions/22737#discussioncomment-9478727

> I've now updated to cover some more things, and considering the
> possibility of multiple guest addresses..  Turns out etherpad doesn't
> really do tables, so it's two sections for the two suggested modes,
> with matching subheadings.

It does, but I disabled the plug-in as you reported an issue which
turned out to be https://github.com/bitwarden/clients/issues/17598
instead, and I was trying to sort out other possible reasons.

I just re-enabled it, tables are available from the toolbar, there's
an icon just left of "Font Family". Note that it's still beta:

  https://www.npmjs.com/package/ep_data_tables

and it has a couple of glitches. I just found one (which I didn't debug
or report yet): don't start a page with a table, always write something
before, otherwise it gets duplicated every time you load the document.

Other than that it looks reasonably robust to me, maybe quickly try with
a test pad first but I think it should be usable.

-- 
Stefano

Re: Thoughts on interface modes / multiple guest addresses

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 5722 bytes --]

On Wed, Dec 17, 2025 at 01:29:36AM +0100, Stefano Brivio wrote:
> On Tue, 16 Dec 2025 16:53:49 +1100
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > Hi Jon,
> > 
> > As discussed on the call yesterday, I've written up my thoughts on
> > what a bunch of the address semantics should be.  Turns out I'd
> > already done some of this at:
> >     https://pad.passt.top/p/InterfaceMode
> 
> Two general comments:
> 
> 1. local mode is already implemented, and some things such as the
>    interface name ("tap0") are already defined, see man page and
>    'pasta -- pasta --config-net ip a'

Yes, I'm aware.  The two modes have the "normal" and "local" local to
indicate the existing modes they're more similar to (neither is
identical to current behaviour).  Another point I left out is that
this is intended as an endpoint to aim at.  Getting there I expect to
involve extra stuff for compatiblity along the way.

> 2. I think it's more relevant to define the basics of how one switches
>    between the existing local mode and a mode where we copy addresses
>    and routes (as they become available on the host), rather than
>    defining every single detail of these modes.

Oh.. right.  I guess I did't make this clear: these are modes set by
the command line (details TBD), we never switch between them at
runtime.  They kind of have to be, because which mode we're in affects
how we respond to runtime changes.

>    In these terms, I think it would actually be helpful to *avoid*
>    seeing them as separate modes. If there's no host connectivity, we'll
>    start in local mode, and switch to the other mode as we get addresses
>    and routes configured... just to switch back to the previous mode if
>    we lose them.

The whole point of all-interface mode (better name suggestions
welcome) is that the guest's routing configuration *doesn't* change,
even if the host's does.

Advantages:
 * The host can have whatever source-dependent, multi-path, bizarro
   routing configuration, changing as often as it likes and we (and
   the guest) don't care.  Guest routes to the host, host routes
   onward from there.
 * We have a consistent (link local) way of addressing the host
   regardless of what changes happen on the host side
 * We have the freedom to allocate link-local addresses if we want
   them for any purpose
Disadvantages:
 * No access to external peers via link-local address
 * Guest's routing setup is visibly different from the host's (so less
   L3 transparent)

I actually think that's a more useful and robust way of operating for
most use cases and we should eventually make it the default.
One-interface mode is for the use cases where those disadvantages are
fatal.

There is another possible option here: present multiple interfaces in
the guest, one for each host interface.  I'm not including it, since
it's basically equivalent to having multiple pasta instances in
one-interface mode.  To implement this, we'd basically have to
implement one-interface mode first.

>    So does it really help to have "modes" instead of just considering
>    what addresses and routes are we going to delete, and when? Because
>    that's what we'll need to do anyway (and that's what I think defines
>    the design).

I haven't seen a way to define coherent semantics that cover all the
use cases without introducing two modes.  The overlapping constraints
here are:

 * With passt or !--config-net, we can't fully control the guest's
   networking config.  We both can't set things with arbitrary
   precision, and we don't have a way of forcing an update when things
   change.
 * If we're dealing with multiple host interfaces - either
   concurrently or to a lesser extent over time, then there's no way
   to coherently map host-side link-local addresses to the guest.

>    I see that this is not an explicit use case in Jon's list (which I
>    still have to review), but it's one of the most two fundamental ones
>    I think (that, and Podman Quadlets), also nicely described by a user
>    at:
> 
>      https://github.com/containers/podman/discussions/22737#discussioncomment-9478727

Ah, yes, that is another case.   I think it would work out equivalent
to one-interface mode attached to a dummy0 interface on the host.  So,
it should be fairly easy to implement in terms of one-interface mode,
just pretending a dummy0 existed even if it doesn't.

> > I've now updated to cover some more things, and considering the
> > possibility of multiple guest addresses..  Turns out etherpad doesn't
> > really do tables, so it's two sections for the two suggested modes,
> > with matching subheadings.
> 
> It does, but I disabled the plug-in as you reported an issue which
> turned out to be https://github.com/bitwarden/clients/issues/17598
> instead, and I was trying to sort out other possible reasons.
> 
> I just re-enabled it, tables are available from the toolbar, there's
> an icon just left of "Font Family". Note that it's still beta:
> 
>   https://www.npmjs.com/package/ep_data_tables
> 
> and it has a couple of glitches. I just found one (which I didn't debug
> or report yet): don't start a page with a table, always write something
> before, otherwise it gets duplicated every time you load the document.
> 
> Other than that it looks reasonably robust to me, maybe quickly try with
> a test pad first but I think it should be usable.

Great, thanks.

-- 
David Gibson (he or they)	| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you, not the other way
				| around.
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Thoughts on interface modes / multiple guest addresses

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1800 bytes --]

On Wed, Dec 17, 2025 at 01:01:36PM +1100, David Gibson wrote:
> On Wed, Dec 17, 2025 at 01:29:36AM +0100, Stefano Brivio wrote:
> > On Tue, 16 Dec 2025 16:53:49 +1100
> > David Gibson <david@gibson.dropbear.id.au> wrote:
[snip]
> > > I've now updated to cover some more things, and considering the
> > > possibility of multiple guest addresses..  Turns out etherpad doesn't
> > > really do tables, so it's two sections for the two suggested modes,
> > > with matching subheadings.
> > 
> > It does, but I disabled the plug-in as you reported an issue which
> > turned out to be https://github.com/bitwarden/clients/issues/17598
> > instead, and I was trying to sort out other possible reasons.
> > 
> > I just re-enabled it, tables are available from the toolbar, there's
> > an icon just left of "Font Family". Note that it's still beta:
> > 
> >   https://www.npmjs.com/package/ep_data_tables
> > 
> > and it has a couple of glitches. I just found one (which I didn't debug
> > or report yet): don't start a page with a table, always write something
> > before, otherwise it gets duplicated every time you load the document.
> > 
> > Other than that it looks reasonably robust to me, maybe quickly try with
> > a test pad first but I think it should be usable.
> 
> Great, thanks.

It definitely has some jank - sometimes cursor / delete / backspace
don't go where you expect.  It's usable, though.

I've updated this page to use a table.  I've also added what the
current behaviour is (both "local mode" and normal) for comparison,
along with some other revisions.

-- 
David Gibson (he or they)	| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you, not the other way
				| around.
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Thoughts on interface modes / multiple guest addresses

[Stefano Brivio]

On Wed, 17 Dec 2025 16:00:38 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> On Wed, Dec 17, 2025 at 01:01:36PM +1100, David Gibson wrote:
> > On Wed, Dec 17, 2025 at 01:29:36AM +0100, Stefano Brivio wrote:  
> > > On Tue, 16 Dec 2025 16:53:49 +1100
> > > David Gibson <david@gibson.dropbear.id.au> wrote:  
> [snip]
> > > > I've now updated to cover some more things, and considering the
> > > > possibility of multiple guest addresses..  Turns out etherpad doesn't
> > > > really do tables, so it's two sections for the two suggested modes,
> > > > with matching subheadings.  
> > > 
> > > It does, but I disabled the plug-in as you reported an issue which
> > > turned out to be https://github.com/bitwarden/clients/issues/17598
> > > instead, and I was trying to sort out other possible reasons.
> > > 
> > > I just re-enabled it, tables are available from the toolbar, there's
> > > an icon just left of "Font Family". Note that it's still beta:
> > > 
> > >   https://www.npmjs.com/package/ep_data_tables
> > > 
> > > and it has a couple of glitches. I just found one (which I didn't debug
> > > or report yet): don't start a page with a table, always write something
> > > before, otherwise it gets duplicated every time you load the document.
> > > 
> > > Other than that it looks reasonably robust to me, maybe quickly try with
> > > a test pad first but I think it should be usable.  
> > 
> > Great, thanks.  
> 
> It definitely has some jank - sometimes cursor / delete / backspace
> don't go where you expect.  It's usable, though.

Unfortunately as soon as Jon loaded it (or was it me?) the table
started getting duplicated empty rows and became unusable, just the same
behaviour as I hit when I had no text before it.

I converted it to a monospaced ASCII table, not elegant but that will
have to do for the moment.

> I've updated this page to use a table.  I've also added what the
> current behaviour is (both "local mode" and normal) for comparison,
> along with some other revisions.

Now, while I don't think any of this is relevant for the netlink monitor
(but I didn't have a chance to comment about that yet), I had a very
quick look anyway and I spotted a few inaccuracies.

For example, we definitely don't set DHCP option 2, and in some cases we
do send DHCP option 121. On top of that, I don't think you can
deprecate options without offering equivalent functionality. But anyway,
not an actual review.

-- 
Stefano

Re: Thoughts on interface modes / multiple guest addresses

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 3241 bytes --]

On Thu, Dec 18, 2025 at 12:03:44AM +0100, Stefano Brivio wrote:
> On Wed, 17 Dec 2025 16:00:38 +1100
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > On Wed, Dec 17, 2025 at 01:01:36PM +1100, David Gibson wrote:
> > > On Wed, Dec 17, 2025 at 01:29:36AM +0100, Stefano Brivio wrote:  
> > > > On Tue, 16 Dec 2025 16:53:49 +1100
> > > > David Gibson <david@gibson.dropbear.id.au> wrote:  
> > [snip]
> > > > > I've now updated to cover some more things, and considering the
> > > > > possibility of multiple guest addresses..  Turns out etherpad doesn't
> > > > > really do tables, so it's two sections for the two suggested modes,
> > > > > with matching subheadings.  
> > > > 
> > > > It does, but I disabled the plug-in as you reported an issue which
> > > > turned out to be https://github.com/bitwarden/clients/issues/17598
> > > > instead, and I was trying to sort out other possible reasons.
> > > > 
> > > > I just re-enabled it, tables are available from the toolbar, there's
> > > > an icon just left of "Font Family". Note that it's still beta:
> > > > 
> > > >   https://www.npmjs.com/package/ep_data_tables
> > > > 
> > > > and it has a couple of glitches. I just found one (which I didn't debug
> > > > or report yet): don't start a page with a table, always write something
> > > > before, otherwise it gets duplicated every time you load the document.
> > > > 
> > > > Other than that it looks reasonably robust to me, maybe quickly try with
> > > > a test pad first but I think it should be usable.  
> > > 
> > > Great, thanks.  
> > 
> > It definitely has some jank - sometimes cursor / delete / backspace
> > don't go where you expect.  It's usable, though.
> 
> Unfortunately as soon as Jon loaded it (or was it me?) the table
> started getting duplicated empty rows and became unusable, just the same
> behaviour as I hit when I had no text before it.

Well, poop.  I guess it's not usable, after all.

> I converted it to a monospaced ASCII table, not elegant but that will
> have to do for the moment.

Ok.

> > I've updated this page to use a table.  I've also added what the
> > current behaviour is (both "local mode" and normal) for comparison,
> > along with some other revisions.
> 
> Now, while I don't think any of this is relevant for the netlink monitor
> (but I didn't have a chance to comment about that yet), I had a very
> quick look anyway and I spotted a few inaccuracies.

I think it is relevant, because it's two different approaches to
handling what we do when we get updates from it.

> For example, we definitely don't set DHCP option 2,

Oops, I meant option 3.  Corrected now.

> and in some cases we
> do send DHCP option 121.

Ah, yes.  Corrected.

> On top of that, I don't think you can
> deprecate options without offering equivalent functionality. But anyway,
> not an actual review.

We absolutely could (and occasionally have), but it does require
careful thought about the tradeoffs.  Again, it's a draft.

-- 
David Gibson (he or they)	| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you, not the other way
				| around.
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Thoughts on interface modes / multiple guest addresses

[Jon Maloy]

On 2025-12-16 21:01, David Gibson wrote:
> On Wed, Dec 17, 2025 at 01:29:36AM +0100, Stefano Brivio wrote:
>> On Tue, 16 Dec 2025 16:53:49 +1100
>> David Gibson <david@gibson.dropbear.id.au> wrote:
>>
>>> Hi Jon,
>>>
>>> As discussed on the call yesterday, I've written up my thoughts on
>>> what a bunch of the address semantics should be.  Turns out I'd
>>> already done some of this at:
>>>      https://pad.passt.top/p/InterfaceMode
>>
>> Two general comments:
>>
>> 1. local mode is already implemented, and some things such as the
>>     interface name ("tap0") are already defined, see man page and
>>     'pasta -- pasta --config-net ip a'
> 
> Yes, I'm aware.  The two modes have the "normal" and "local" local to
> indicate the existing modes they're more similar to (neither is
> identical to current behaviour).  Another point I left out is that
> this is intended as an endpoint to aim at.  Getting there I expect to
> involve extra stuff for compatiblity along the way.
> 
>> 2. I think it's more relevant to define the basics of how one switches
>>     between the existing local mode and a mode where we copy addresses
>>     and routes (as they become available on the host), rather than
>>     defining every single detail of these modes.
> 
> Oh.. right.  I guess I did't make this clear: these are modes set by
> the command line (details TBD), we never switch between them at
> runtime.  They kind of have to be, because which mode we're in affects
> how we respond to runtime changes.
> 
>>     In these terms, I think it would actually be helpful to *avoid*
>>     seeing them as separate modes. If there's no host connectivity, we'll
>>     start in local mode, and switch to the other mode as we get addresses
>>     and routes configured... just to switch back to the previous mode if
>>     we lose them.
> 
> The whole point of all-interface mode (better name suggestions
> welcome) is that the guest's routing configuration *doesn't* change,
> even if the host's does.
> 
> Advantages:
>   * The host can have whatever source-dependent, multi-path, bizarro
>     routing configuration, changing as often as it likes and we (and
>     the guest) don't care.  Guest routes to the host, host routes
>     onward from there.

And the other way around, the guest should be able to set up whatever
he wants if that is his choice, without affecting the host. Maybe
even multiple interfaces, for whatever reason that might be.

We could have two modes: "transparent" and "opaque"

In opaque mode we get basically what you describe above, plus that we 
allow the guest to add new addresses/routes in runtime.
So, we keep the multi-address configuration and and the nameaspace side
subscription, but block host-side subscriptions in this mode.

Conversely, we are fully open to host-changes in transparent mode, we
subscribe for host-side changes, forward those to the namespace,
However, we don't allow the guest user to manipulate anything in run-time.

Hybrid modes might be possible, e.g., that we allow one stable link 
local address even in transparent mode.

Would that make sense?

>   * We have a consistent (link local) way of addressing the host
>     regardless of what changes happen on the host side
>   * We have the freedom to allocate link-local addresses if we want
>     them for any purpose
> Disadvantages:
>   * No access to external peers via link-local address
>   * Guest's routing setup is visibly different from the host's (so less
>     L3 transparent)
> 
> I actually think that's a more useful and robust way of operating for
> most use cases and we should eventually make it the default.
> One-interface mode is for the use cases where those disadvantages are
> fatal.
> 
> There is another possible option here: present multiple interfaces in
> the guest, one for each host interface. 

In transparent mode that would be a natural further step.

///jon

> I'm not including it, since
> it's basically equivalent to having multiple pasta instances in
> one-interface mode.  To implement this, we'd basically have to
> implement one-interface mode first.
> 
>>     So does it really help to have "modes" instead of just considering
>>     what addresses and routes are we going to delete, and when? Because
>>     that's what we'll need to do anyway (and that's what I think defines
>>     the design).
> 
> I haven't seen a way to define coherent semantics that cover all the
> use cases without introducing two modes.  The overlapping constraints
> here are:
> 
>   * With passt or !--config-net, we can't fully control the guest's
>     networking config.  We both can't set things with arbitrary
>     precision, and we don't have a way of forcing an update when things
>     change.
>   * If we're dealing with multiple host interfaces - either
>     concurrently or to a lesser extent over time, then there's no way
>     to coherently map host-side link-local addresses to the guest.
> 
>>     I see that this is not an explicit use case in Jon's list (which I
>>     still have to review), but it's one of the most two fundamental ones
>>     I think (that, and Podman Quadlets), also nicely described by a user
>>     at:
>>
>>       https://github.com/containers/podman/discussions/22737#discussioncomment-9478727
> 
> Ah, yes, that is another case.   I think it would work out equivalent
> to one-interface mode attached to a dummy0 interface on the host.  So,
> it should be fairly easy to implement in terms of one-interface mode,
> just pretending a dummy0 existed even if it doesn't.
> 
>>> I've now updated to cover some more things, and considering the
>>> possibility of multiple guest addresses..  Turns out etherpad doesn't
>>> really do tables, so it's two sections for the two suggested modes,
>>> with matching subheadings.
>>
>> It does, but I disabled the plug-in as you reported an issue which
>> turned out to be https://github.com/bitwarden/clients/issues/17598
>> instead, and I was trying to sort out other possible reasons.
>>
>> I just re-enabled it, tables are available from the toolbar, there's
>> an icon just left of "Font Family". Note that it's still beta:
>>
>>    https://www.npmjs.com/package/ep_data_tables
>>
>> and it has a couple of glitches. I just found one (which I didn't debug
>> or report yet): don't start a page with a table, always write something
>> before, otherwise it gets duplicated every time you load the document.
>>
>> Other than that it looks reasonably robust to me, maybe quickly try with
>> a test pad first but I think it should be usable.
> 
> Great, thanks.
> 

Re: Thoughts on interface modes / multiple guest addresses

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 8899 bytes --]

On Wed, Dec 17, 2025 at 03:01:31PM -0500, Jon Maloy wrote:
> 
> 
> On 2025-12-16 21:01, David Gibson wrote:
> > On Wed, Dec 17, 2025 at 01:29:36AM +0100, Stefano Brivio wrote:
> > > On Tue, 16 Dec 2025 16:53:49 +1100
> > > David Gibson <david@gibson.dropbear.id.au> wrote:
> > > 
> > > > Hi Jon,
> > > > 
> > > > As discussed on the call yesterday, I've written up my thoughts on
> > > > what a bunch of the address semantics should be.  Turns out I'd
> > > > already done some of this at:
> > > >      https://pad.passt.top/p/InterfaceMode
> > > 
> > > Two general comments:
> > > 
> > > 1. local mode is already implemented, and some things such as the
> > >     interface name ("tap0") are already defined, see man page and
> > >     'pasta -- pasta --config-net ip a'
> > 
> > Yes, I'm aware.  The two modes have the "normal" and "local" local to
> > indicate the existing modes they're more similar to (neither is
> > identical to current behaviour).  Another point I left out is that
> > this is intended as an endpoint to aim at.  Getting there I expect to
> > involve extra stuff for compatiblity along the way.
> > 
> > > 2. I think it's more relevant to define the basics of how one switches
> > >     between the existing local mode and a mode where we copy addresses
> > >     and routes (as they become available on the host), rather than
> > >     defining every single detail of these modes.
> > 
> > Oh.. right.  I guess I did't make this clear: these are modes set by
> > the command line (details TBD), we never switch between them at
> > runtime.  They kind of have to be, because which mode we're in affects
> > how we respond to runtime changes.
> > 
> > >     In these terms, I think it would actually be helpful to *avoid*
> > >     seeing them as separate modes. If there's no host connectivity, we'll
> > >     start in local mode, and switch to the other mode as we get addresses
> > >     and routes configured... just to switch back to the previous mode if
> > >     we lose them.
> > 
> > The whole point of all-interface mode (better name suggestions
> > welcome) is that the guest's routing configuration *doesn't* change,
> > even if the host's does.
> > 
> > Advantages:
> >   * The host can have whatever source-dependent, multi-path, bizarro
> >     routing configuration, changing as often as it likes and we (and
> >     the guest) don't care.  Guest routes to the host, host routes
> >     onward from there.
> 
> And the other way around, the guest should be able to set up whatever
> he wants if that is his choice, without affecting the host. Maybe
> even multiple interfaces, for whatever reason that might be.

I'm not sure I follow.  We can't change net config on the host, so
that's basically always true - in both proposed modes, and right now.
So maybe you're meaning something different?  I'm not sure what,
though.

> We could have two modes: "transparent" and "opaque"

I'm not 100% sold, but I think those are at least a bit better than my
current names (doc updated).

> In opaque mode we get basically what you describe above, plus that we allow
> the guest to add new addresses/routes in runtime.

THe guest could always add extra routes at runtime.  Adding extra
addresses... well, it always could, the question is what - if anything
- passt/pasta should do about it.  What do you propose?

> So, we keep the multi-address configuration and and the nameaspace side
> subscription, but block host-side subscriptions in this mode.

My intention was that we don't use a host-side *route* subscription in
this mode.  We would still need a host-side address subscription to
implement '-a auto', which I was proposing to have allowed (and
enabled by default) in opaque mode.

> Conversely, we are fully open to host-changes in transparent mode, we
> subscribe for host-side changes, forward those to the namespace,

With --config-net, yes.  For passt and !--config-net we can't control
the guest configuration; we can only reflect as much of the host state
as possible via DHCP and NDP, and hope the guest will consult that and
update itself.

> However, we don't allow the guest user to manipulate anything in run-time.

Well, we can't really prevent it.  I guess you mean if the guest
reconfigures itself, then things might break entirely and it's not our
fault?

> Hybrid modes might be possible,

I haven't (so far) seen a sensible way to hybridise these models.  The
main sticking point is giving a consistent meaning to link-local
addresses when used by the guest.

> e.g., that we allow one stable link local
> address even in transparent mode.

That's allowed by the proposed model: transparent mode with '-a <ll
addr> -a auto'.

> Would that make sense?
> 
> >   * We have a consistent (link local) way of addressing the host
> >     regardless of what changes happen on the host side
> >   * We have the freedom to allocate link-local addresses if we want
> >     them for any purpose
> > Disadvantages:
> >   * No access to external peers via link-local address
> >   * Guest's routing setup is visibly different from the host's (so less
> >     L3 transparent)
> > 
> > I actually think that's a more useful and robust way of operating for
> > most use cases and we should eventually make it the default.
> > One-interface mode is for the use cases where those disadvantages are
> > fatal.
> > 
> > There is another possible option here: present multiple interfaces in
> > the guest, one for each host interface.
> 
> In transparent mode that would be a natural further step.

Yes.  Supporting multiple guest side pifs is a preqrequisite, though.
I'm working towards that, but it's a fair way off.

> 
> ///jon
> 
> > I'm not including it, since
> > it's basically equivalent to having multiple pasta instances in
> > one-interface mode.  To implement this, we'd basically have to
> > implement one-interface mode first.
> > 
> > >     So does it really help to have "modes" instead of just considering
> > >     what addresses and routes are we going to delete, and when? Because
> > >     that's what we'll need to do anyway (and that's what I think defines
> > >     the design).
> > 
> > I haven't seen a way to define coherent semantics that cover all the
> > use cases without introducing two modes.  The overlapping constraints
> > here are:
> > 
> >   * With passt or !--config-net, we can't fully control the guest's
> >     networking config.  We both can't set things with arbitrary
> >     precision, and we don't have a way of forcing an update when things
> >     change.
> >   * If we're dealing with multiple host interfaces - either
> >     concurrently or to a lesser extent over time, then there's no way
> >     to coherently map host-side link-local addresses to the guest.
> > 
> > >     I see that this is not an explicit use case in Jon's list (which I
> > >     still have to review), but it's one of the most two fundamental ones
> > >     I think (that, and Podman Quadlets), also nicely described by a user
> > >     at:
> > > 
> > >       https://github.com/containers/podman/discussions/22737#discussioncomment-9478727
> > 
> > Ah, yes, that is another case.   I think it would work out equivalent
> > to one-interface mode attached to a dummy0 interface on the host.  So,
> > it should be fairly easy to implement in terms of one-interface mode,
> > just pretending a dummy0 existed even if it doesn't.
> > 
> > > > I've now updated to cover some more things, and considering the
> > > > possibility of multiple guest addresses..  Turns out etherpad doesn't
> > > > really do tables, so it's two sections for the two suggested modes,
> > > > with matching subheadings.
> > > 
> > > It does, but I disabled the plug-in as you reported an issue which
> > > turned out to be https://github.com/bitwarden/clients/issues/17598
> > > instead, and I was trying to sort out other possible reasons.
> > > 
> > > I just re-enabled it, tables are available from the toolbar, there's
> > > an icon just left of "Font Family". Note that it's still beta:
> > > 
> > >    https://www.npmjs.com/package/ep_data_tables
> > > 
> > > and it has a couple of glitches. I just found one (which I didn't debug
> > > or report yet): don't start a page with a table, always write something
> > > before, otherwise it gets duplicated every time you load the document.
> > > 
> > > Other than that it looks reasonably robust to me, maybe quickly try with
> > > a test pad first but I think it should be usable.
> > 
> > Great, thanks.
> > 
> 

-- 
David Gibson (he or they)	| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you, not the other way
				| around.
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Thoughts on interface modes / multiple guest addresses

[Stefano Brivio]

On Wed, 17 Dec 2025 13:01:36 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> On Wed, Dec 17, 2025 at 01:29:36AM +0100, Stefano Brivio wrote:
> > On Tue, 16 Dec 2025 16:53:49 +1100
> > David Gibson <david@gibson.dropbear.id.au> wrote:
> >   
> > > Hi Jon,
> > > 
> > > As discussed on the call yesterday, I've written up my thoughts on
> > > what a bunch of the address semantics should be.  Turns out I'd
> > > already done some of this at:
> > >     https://pad.passt.top/p/InterfaceMode  
> > 
> > Two general comments:
> > 
> > 1. local mode is already implemented, and some things such as the
> >    interface name ("tap0") are already defined, see man page and
> >    'pasta -- pasta --config-net ip a'  
> 
> Yes, I'm aware.  The two modes have the "normal" and "local" local to
> indicate the existing modes they're more similar to (neither is
> identical to current behaviour).  Another point I left out is that
> this is intended as an endpoint to aim at.  Getting there I expect to
> involve extra stuff for compatiblity along the way.
> 
> > 2. I think it's more relevant to define the basics of how one switches
> >    between the existing local mode and a mode where we copy addresses
> >    and routes (as they become available on the host), rather than
> >    defining every single detail of these modes.  
> 
> Oh.. right.  I guess I did't make this clear: these are modes set by
> the command line (details TBD), we never switch between them at
> runtime.  They kind of have to be, because which mode we're in affects
> how we respond to runtime changes.
> 
> >    In these terms, I think it would actually be helpful to *avoid*
> >    seeing them as separate modes. If there's no host connectivity, we'll
> >    start in local mode, and switch to the other mode as we get addresses
> >    and routes configured... just to switch back to the previous mode if
> >    we lose them.  
> 
> The whole point of all-interface mode (better name suggestions
> welcome) is that the guest's routing configuration *doesn't* change,
> even if the host's does.

If that's what you mean, I think we're talking about two rather
different things.

At the moment, local mode is used if a given IP version doesn't have a
configured address on the host, in that specific moment. This is what
the netlink monitor needs to make independent of the timing.

But if the user wants to specify addresses, or routes, we certainly
don't want to override them. So we would rather have an "override" or
"manual" mode (same as we already have now) as opposed to a "template"
or "automatic" mode where we copy (at runtime, with a monitor)
addresses and routes.

It's actually rather simple to implement a netlink monitor on top of
this, and that's for sure what we need to implement because it's rather
broken in Podman and rootlesskit and it has been for a while.

I think that's the priority and what Jon was about to do rather than
spending now months to revise addressing and all possible defaults and
command line options. I mean, feel free to do that of course but I
think it needs to be implemented in parallel at that point.

> Advantages:
>  * The host can have whatever source-dependent, multi-path, bizarro
>    routing configuration, changing as often as it likes and we (and
>    the guest) don't care.  Guest routes to the host, host routes
>    onward from there.
>  * We have a consistent (link local) way of addressing the host
>    regardless of what changes happen on the host side
>  * We have the freedom to allocate link-local addresses if we want
>    them for any purpose
> Disadvantages:
>  * No access to external peers via link-local address
>  * Guest's routing setup is visibly different from the host's (so less
>    L3 transparent)
> 
> I actually think that's a more useful and robust way of operating for
> most use cases and we should eventually make it the default.

Making things look like they're on the host is a very fundamental
feature that many users appreciate and use.

I haven't reviewed the rest in detail but forcing users to specify a
new option to go back to a convenient default they had for years now
doesn't sound like a good idea.

> One-interface mode is for the use cases where those disadvantages are
> fatal.
> 
> There is another possible option here: present multiple interfaces in
> the guest, one for each host interface.  I'm not including it, since
> it's basically equivalent to having multiple pasta instances in
> one-interface mode.  To implement this, we'd basically have to
> implement one-interface mode first.
> 
> >    So does it really help to have "modes" instead of just considering
> >    what addresses and routes are we going to delete, and when? Because
> >    that's what we'll need to do anyway (and that's what I think defines
> >    the design).  
> 
> I haven't seen a way to define coherent semantics that cover all the
> use cases without introducing two modes.  The overlapping constraints
> here are:

The two modes could be what we roughly have now: -a / -g imply that we
don't copy addresses / routes, and we don't source them for DHCP /
SLAAC / DHCPv6 either. The netlink monitor could simply be enabled when
the user doesn't override things.

>  * With passt or !--config-net, we can't fully control the guest's
>    networking config.  We both can't set things with arbitrary
>    precision, and we don't have a way of forcing an update when things
>    change.

What is the problem with that? Nobody reported any issue with it. It's
actually expected.

>  * If we're dealing with multiple host interfaces - either
>    concurrently or to a lesser extent over time, then there's no way
>    to coherently map host-side link-local addresses to the guest.

Also never reported as an issue as far as I know.

> >    I see that this is not an explicit use case in Jon's list (which I
> >    still have to review), but it's one of the most two fundamental ones
> >    I think (that, and Podman Quadlets), also nicely described by a user
> >    at:
> > 
> >      https://github.com/containers/podman/discussions/22737#discussioncomment-9478727  
> 
> Ah, yes, that is another case.   I think it would work out equivalent
> to one-interface mode attached to a dummy0 interface on the host.  So,
> it should be fairly easy to implement in terms of one-interface mode,
> just pretending a dummy0 existed even if it doesn't.

That's pretty much the most important / most frequently reported case,
not just another case. But anyway, there's no need for dummy0 or
suchlike, we can just keep what we use as "local mode" (which wouldn't
be a mode anymore, rather a temporary state).

-- 
Stefano

Re: Thoughts on interface modes / multiple guest addresses

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 13140 bytes --]

On Thu, Dec 18, 2025 at 12:22:20AM +0100, Stefano Brivio wrote:
> On Wed, 17 Dec 2025 13:01:36 +1100
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > On Wed, Dec 17, 2025 at 01:29:36AM +0100, Stefano Brivio wrote:
> > > On Tue, 16 Dec 2025 16:53:49 +1100
> > > David Gibson <david@gibson.dropbear.id.au> wrote:
> > >   
> > > > Hi Jon,
> > > > 
> > > > As discussed on the call yesterday, I've written up my thoughts on
> > > > what a bunch of the address semantics should be.  Turns out I'd
> > > > already done some of this at:
> > > >     https://pad.passt.top/p/InterfaceMode  
> > > 
> > > Two general comments:
> > > 
> > > 1. local mode is already implemented, and some things such as the
> > >    interface name ("tap0") are already defined, see man page and
> > >    'pasta -- pasta --config-net ip a'  
> > 
> > Yes, I'm aware.  The two modes have the "normal" and "local" local to
> > indicate the existing modes they're more similar to (neither is
> > identical to current behaviour).  Another point I left out is that
> > this is intended as an endpoint to aim at.  Getting there I expect to
> > involve extra stuff for compatiblity along the way.
> > 
> > > 2. I think it's more relevant to define the basics of how one switches
> > >    between the existing local mode and a mode where we copy addresses
> > >    and routes (as they become available on the host), rather than
> > >    defining every single detail of these modes.  
> > 
> > Oh.. right.  I guess I did't make this clear: these are modes set by
> > the command line (details TBD), we never switch between them at
> > runtime.  They kind of have to be, because which mode we're in affects
> > how we respond to runtime changes.
> > 
> > >    In these terms, I think it would actually be helpful to *avoid*
> > >    seeing them as separate modes. If there's no host connectivity, we'll
> > >    start in local mode, and switch to the other mode as we get addresses
> > >    and routes configured... just to switch back to the previous mode if
> > >    we lose them.  
> > 
> > The whole point of all-interface mode (better name suggestions
> > welcome) is that the guest's routing configuration *doesn't* change,
> > even if the host's does.
> 
> If that's what you mean, I think we're talking about two rather
> different things.

Evidently.  This is my attempt to convey the picture in my head, I
don't yet have much grasp of the picture in your head.

> At the moment, local mode is used if a given IP version doesn't have a
> configured address on the host, in that specific moment. This is what
> the netlink monitor needs to make independent of the timing.

Right.  Maybe labelling this "local mode" was misleading.  I'm using
that because how we'd implement it is pretty close to how the current
"local mode" works.

> But if the user wants to specify addresses, or routes, we certainly
> don't want to override them. So we would rather have an "override" or
> "manual" mode (same as we already have now) as opposed to a "template"
> or "automatic" mode where we copy (at runtime, with a monitor)
> addresses and routes.

Ah, yes, good.  There are less differences here than I thought, so
I've removed several rows from the table, hooray!

> It's actually rather simple to implement a netlink monitor on top of
> this, and that's for sure what we need to implement because it's rather
> broken in Podman and rootlesskit and it has been for a while.

Many things have been "rather simple to implement" until I actually
did them.

> I think that's the priority and what Jon was about to do rather than
> spending now months to revise addressing and all possible defaults and
> command line options. I mean, feel free to do that of course but I
> think it needs to be implemented in parallel at that point.

I'm really trying not to get into the weeds here.  But I think through
what the semantics of the new feature should be, and then weeds are
all around me.

For multi-address support there are at least four things to consider:

(a) What goes in our internal list of addresses to give the guest?

    a.1. Everything listed with -a?
    a.2. Everything on the host?
    a.3. Everything on the host template interface?
    a.4. A link local address we pick?
    a.5. Some combination of the above.

Unlike routes (that I can see), I'm pretty sure there are use cases
where we want both host-copied addresses (for transparency) and
explicit addresses (for a stable way of communicating with the host).

-a auto (orthogonal to the mode proposals) is a suggestion of a way to
make that configurable.  Maybe it should be -a <ifname> ir -a all, to
cover a.3?

(b) How do we convey those addresses to the guest?

With --config-net it's fairly straightforward: whenever something
changes in the internal list add/remove it in the guest.  Without
--config-net (including passt) it's trickier.

    b.1. We can use DHCP / RA lease times
        - Q: How long should they be?
    b.2. We can use DHCPNAK / RECONFIGURE
        - Can we count on the guest responding to these properly?
	- Do we care?

(c) What if the guest uses an address we didn't tell it?

That is, a multi-address equivalent of addr_seen.

   c.1. Add observed guest addresses to internal list
   	- closest equivalent to current addr_seen
	- need to flag them so we don't send them back to the guest
	- Q: for how long do we keep them?
   c.2. Guest side netlink monitor updates 
   	- Can't be used with passt
	- Shouldn't (IMO) be used with !--config-net
   	- Does this give us anything c.1 doesn't?
   c.3. Discard packets with an address not in the internal list
	- Simplest semantics
   	- Breaking change
   c.4. Send a DHCPNAK if the guest uses an unexpected address
        - Combined with?

(d) Interaction between addresses and routes

If we remove an address, and that address prefix was responsible for
the guest route to reach $GGW, we need to update routes as well.

> > Advantages:
> >  * The host can have whatever source-dependent, multi-path, bizarro
> >    routing configuration, changing as often as it likes and we (and
> >    the guest) don't care.  Guest routes to the host, host routes
> >    onward from there.
> >  * We have a consistent (link local) way of addressing the host
> >    regardless of what changes happen on the host side
> >  * We have the freedom to allocate link-local addresses if we want
> >    them for any purpose
> > Disadvantages:
> >  * No access to external peers via link-local address
> >  * Guest's routing setup is visibly different from the host's (so less
> >    L3 transparent)
> > 
> > I actually think that's a more useful and robust way of operating for
> > most use cases and we should eventually make it the default.
> 
> Making things look like they're on the host is a very fundamental
> feature that many users appreciate and use.

There are degrees to "looking like the host".  We'll never be perfect,
so we have some choice on which aspects matter.

Sharing the host address makes a lot of things work smoother, and I
think we absolutely should keep that by default.

I'm pretty sure *far* fewer things care whether the host's routing
config is mirrored.  That's why I'm suggesting an approach that
preserves the former while dropping the latter.


Part of the issue here is that if the host has multiple links, we
really can't "look like the host" without putting multiple links in
the guest.  There's really no way to make a single-link config look
like a multi-link config.  Multiple guest interfaces is feasible for
pasta (though a fair bit of work), but pretty awkward for passt.

> I haven't reviewed the rest in detail but forcing users to specify a
> new option to go back to a convenient default they had for years now
> doesn't sound like a good idea.

Eh, I can easily be persuaded on defaults.

> > One-interface mode is for the use cases where those disadvantages are
> > fatal.
> > 
> > There is another possible option here: present multiple interfaces in
> > the guest, one for each host interface.  I'm not including it, since
> > it's basically equivalent to having multiple pasta instances in
> > one-interface mode.  To implement this, we'd basically have to
> > implement one-interface mode first.
> > 
> > >    So does it really help to have "modes" instead of just considering
> > >    what addresses and routes are we going to delete, and when? Because
> > >    that's what we'll need to do anyway (and that's what I think defines
> > >    the design).  
> > 
> > I haven't seen a way to define coherent semantics that cover all the
> > use cases without introducing two modes.  The overlapping constraints
> > here are:
> 
> The two modes could be what we roughly have now: -a / -g imply that we
> don't copy addresses / routes, and we don't source them for DHCP /
> SLAAC / DHCPv6 either. The netlink monitor could simply be enabled when
> the user doesn't override things.
> 
> >  * With passt or !--config-net, we can't fully control the guest's
> >    networking config.  We both can't set things with arbitrary
> >    precision, and we don't have a way of forcing an update when things
> >    change.
> 
> What is the problem with that? Nobody reported any issue with it. It's
> actually expected.

The problem is that if the guest relies on routing matching the host
to work, then when the host has some configuration we can't or don't
reflect into the guest, it will break.  This already happened with
source-specific routes IIRC.

> >  * If we're dealing with multiple host interfaces - either
> >    concurrently or to a lesser extent over time, then there's no way
> >    to coherently map host-side link-local addresses to the guest.
> 
> Also never reported as an issue as far as I know.

It means whenever I need to write forwarding logic, I don't know what
to do with ll addresses, because there's no clear model what they
mean.

> > >    I see that this is not an explicit use case in Jon's list (which I
> > >    still have to review), but it's one of the most two fundamental ones
> > >    I think (that, and Podman Quadlets), also nicely described by a user
> > >    at:
> > > 
> > >      https://github.com/containers/podman/discussions/22737#discussioncomment-9478727  
> > 
> > Ah, yes, that is another case.   I think it would work out equivalent
> > to one-interface mode attached to a dummy0 interface on the host.  So,
> > it should be fairly easy to implement in terms of one-interface mode,
> > just pretending a dummy0 existed even if it doesn't.
> 
> That's pretty much the most important / most frequently reported case,
> not just another case. But anyway, there's no need for dummy0 or
> suchlike, we can just keep what we use as "local mode" (which wouldn't
> be a mode anymore, rather a temporary state).

Wait... I misread it.  From the reference to -network none, I thought
this one was intended as a permanent state, not a temporary one.  The
laptop-on-a-train case is absolutely the main one I'm thinking about
for opaque mode.  The guest always has an LL address, and the host can
be accessed via it, whether or not the host currently has a network.
When the host does get a network, the guest (maybe) gets extra
addresses and can access the outside world.

A lot of this comes down to: at the moment I can't make sense of what
a "template interface" is or does.  It doesn't affect where packets go
to on the host (that's --outbound-if, except when it isn't).  It
doesn't affect where we packets come from on the host (that's -t and
-u, defaulting to *).  So what the heck does it do?

 - It selects the guest address(es), in a weird indirect way
 - It selects which host interface's neighbours we can talk to with
   link-local addresses.  That's not something are often thinking
   about.
 - It selects the guest interface name, except when it doesn't 
 - Possibly some other things I've forgotten

We try quite hard to pick a template interface, so it's not exactly
optional.  Except it is, because we eventually fall back to local
mode, which doesn't have one.

Making the template interface change - which switching between local
mode and template mode implies - just makes all the above even more
confusing.

The two modes I'm proposing are different directions of resolving this:

"transparent mode" is for when we really do care about what interface
on the host we're using.  We _do_ want to control where packets go to
and come from on the host, and we want to see the details of how
routes and addresses relate to links.

"opaque mode" is when we don't.  We can reach the host, and we might
be able to reach the world via there, but we don't care exactly how.

-- 
David Gibson (he or they)	| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you, not the other way
				| around.
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Thoughts on interface modes / multiple guest addresses

[Stefano Brivio]

On Thu, 18 Dec 2025 14:47:06 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> For multi-address support there are at least four things to consider:

For the bits related https://bugs.passt.top/show_bug.cgi?id=141, I
thought Jon was working on a proposal.

> (a) What goes in our internal list of addresses to give the guest?
> 
>     a.1. Everything listed with -a?

If anything is passed, yes, those, and just those (separately for IP
version), because the user is clearly overriding addresses (as
currently implemented and documented).

>     a.2. Everything on the host?

No, because you can't assume you can configure all those addresses on
a single interface. Adding multiple interfaces is something we could
consider later.

>     a.3. Everything on the host template interface?

Everything on the host template interface if available (as currently
documented).

>     a.4. A link local address we pick?

A link-local address if nothing else is available (as currently
documented). This will need to be permanent for the requirement we
already discussed months ago with Podman developers.

>     a.5. Some combination of the above.
> 
> Unlike routes (that I can see), I'm pretty sure there are use cases
> where we want both host-copied addresses (for transparency) and
> explicit addresses (for a stable way of communicating with the host).

Podman needs some anyway if we start with link-local addresses, to
keep DNS resolution working.

The rest just looks beyond the scope of
https://bugs.passt.top/show_bug.cgi?id=141 to me. I won't stand in the
way of this discussion, of course, but I won't participate either,
because I really don't see it as a priority at the moment.

-- 
Stefano