Re: [PATCH] arp: only send ARP replies for --gateway address

[Nikolay Edigaryev <edigaryev@gmail.com>]

Hello Stefano, I will try to clarify:

I have a single host machine, a dedicated amd64 server, capable of
running multiple Cloud Hypervisor virtual machines backed by /dev/kvm.

I also have a daemon-less CLI software that can provision as many VM
instances as the user wants, e.g. by running "mycli create --kernel
... --disk ... ubuntu".

To run a VM, the user types "mycli run ubuntu", which results in the
creation of two TAP interfaces: one is for passt, one is for Cloud
Hypervisor

"mycli run" then creates a bridge(8) interface, assigns a free IP from
/29 network to it (for example, 10.0.0.3/29), and adds both the TAP
interfaces to that bridge forming up a virtual switch, which allows
passt <-> VM and host <-> communication.

"mycli run ubuntu" also invokes the passt with the following arguments:

>passt --foreground --address 10.0.0.2 --netmask 255.255.255.248 --gateway 10.0.0.1 --mac-addr 52:f1:18:34:28:0b -4 --mtu 1500 --tap-fd 3

Now to the issue: if the user wants to access the VM, for provisioning
purposes, e.g. by running "ssh 10.0.0.2", there's a race between the
real ARP reply from that VM and an ARP reply from passt due to the
code fixed in the patch above.

And even if we add a static ARP entry for that VM on the host, there's
still exist a race on the VM's side.

Here the VM looks up the host's ethernet address and receives one
reply from host (ba:46:4e:27:8b:93) and another from passt
(52:f1:18:34:28:0b):

17:26:42.685718 5a:b7:e3:dc:bb:9f > ba:46:4e:27:8b:93, ethertype ARP
(0x0806), length 42: Request who-has 10.0.0.3 tell 10.0.0.2, length 28
17:26:42.685744 ba:46:4e:27:8b:93 > 5a:b7:e3:dc:bb:9f, ethertype ARP
(0x0806), length 42: Reply 10.0.0.3 is-at ba:46:4e:27:8b:93, length 28
17:26:42.685908 52:f1:18:34:28:0b > 5a:b7:e3:dc:bb:9f, ethertype ARP
(0x0806), length 42: Reply 10.0.0.3 is-at 52:f1:18:34:28:0b, length 28

On Mon, Sep 18, 2023 at 6:01 PM Stefano Brivio <sbrivio@redhat.com> wrote:
>
> On Mon, 18 Sep 2023 12:26:03 +1000
> David Gibson <david@gibson.dropbear.id.au> wrote:
>
> > On Fri, Sep 15, 2023 at 06:20:45PM +0400, Nikolay Edigaryev wrote:
> > > Problem: when passt/pasta are working in a broadcast domain with more
> > > than one host machine,
> >
> > Oof.  So, at present, passt/pasta is really not designed to have more
> > than a single machine on the "tap" side.  Changing the ARP behaviour
> > is likely to be the least of the problems with that setup.
>
> Now I'm confused on which "side" this happens. :) Nikolay, can you
> articulate the issue a bit better? Do you really have multiple *host*
> machines? Does the passt process... move between them?
>
> By the way, the only concern I have with this change is that the guest
> might ignore the gateway address it's being assigned, for whatever
> reason, and by just resolving "almost everything" we guarantee the
> traffic goes out anyway.
>
> If there's no other way to solve the issue you're facing, I would
> rather propose to have this as an option, and perhaps have it off by
> default.
>
> --
> Stefano
>