Re: [PATCH 00/10] RFC/RFT: Optionally copy all routes and addresses for pasta, allow gateway-less routes

[David Gibson <david@gibson.dropbear.id.au>]

[-- Attachment #1: Type: text/plain, Size: 6342 bytes --]

On Sun, May 14, 2023 at 08:14:05PM +0200, Stefano Brivio wrote:
> This series, along with pseudo-related fixes, enables:
> 
> - optional copy of all routes from selected interface in outer
>   namespace, to (hopefully!) fix the issue reported by Callum at:
>     https://github.com/containers/podman/issues/18539
> 
> - optional copy of all addresses, mostly for consistency. It doesn't,
>   however, enable assignment of multiple addresses in the sense
>   requested at:
>     https://bugs.passt.top/show_bug.cgi?id=47
> 
>   because the addresses still need to be present on the host, and
>   the "outer" address isn't selected depending on the address used
>   inside the container
> 
> - operation without a gateway address, to (again, hopefully) support
>   usage of Wireguard endpoints established outside the container,
>     https://bugs.passt.top/show_bug.cgi?id=49
> 
> I tested the single functionalities introduced here, but I didn't
> try to reproduce the setups where the issues were reported, so some
> help with testing is definitely fundamental here. Thanks.

I've sent reviews for some of the simpler patches in this series which
make sense even without the context of the overall aim.  I think those
can be applied immediately.  For the rest of the series, I want to
address the generalities before doing detailed review of the
implementation.

I think the basic idea here is sound: we want to expose anything
routable to the host as routable to the guest, even when the host has
a more complex routing setup that just a netmask on the "main"
interface and a default gateway within that prefix.  But I think we
want to think a bit more deeply about exactly what we need/want to
expose here.

Even with the current code, the default gateway address we advertise
to the guest is kind of meaningless: the guest cannot directly access
that gateway, everything really goes through passt on the host.  This
works because the gateway address (like everything) will ARP/NDP to
passt's host side MAC address and once the packets hit passt it
doesn't matter what the guest thought the routing was going to be.

I think we have a few choices in two more-or-less orthogonal
categories.

A) What routable prefixes do we advertise to the guest?

  A.1) Always a default route (0.0.0.0/0 and ::/0)

We tell the guest that every address is routable via the passt
interface, regardless of routing setup on the host.  This essentially
tells the guest to delegate all routing responsibility to passt.

Advantages:
  * Simple
  * No need to update anything if routing configuration on the host
    changes
Disadvantages:
  * If addresses are unroutable from the host, the guest will only
    know via ICMP/ICMPv6, rather than statically, which may be a worse
    UX on the guest side.  Plus we might need to actually implement
    those host unreachable ICMPs.
  * Might be messy if the guest has multiple interfacees - e.g. if we
    allow passt to be configured to attach to a specific host
    interface only, then we have multiple passts attached to a single
    guest: they'd all be advertising a default route.

  A.2) Copy routable prefixes from the host to the guest

We just advertise those prefixes routable to the host to the guest
(which might include an empty prefix == default route).

Advantages:
  * Guest statically knows what addresses are routable via the passt
    interface
Disadvantages:
  * What do we do with overlapping prefixes?  On the host we might
    have more specific routes pointing to a specific interface.  For
    the guest they all point to the passt interface, so what's the
    point?
  * Can we advertise an arbitrary set of static routes via all our
    mechanisms (--config-net, DHCP, NDP+DHCPv6)?  Even if we can it
    adds more complexity to that code
  * How do we update things if the host routing configuration changes?
  * What do we do if the host has source-based routing or other
    advanced stuff set up?

B) What gateway, if any, do we advertise for each route?

  B.1) Copy it from the host

Advantages:
 * Guest L3 configuration resembles that of the host
Disadvantages:
 * If the host route doesn't have a gateway we have to fall back on
   B.2 or B.3 anyway
 * Misleading: in fact everything is routed by passt and the host
   before it reaches any gateway we're listing here

  B.2) Pick an address to represent passt as gateway

Advantages:
 * Accurately represents that everything is routed by passt
 * We can make this the same as the NAT-to-host address, so we only
   have one "magic" address (per AF)
Disadvantages:
 * Have to allocate an address that's safe, which is tricky (but we
   usually want this for NAT-to-host anyway)
 * Do we want just one address, or one for each distinct gateway from
   the host?
 * If we can't pick something in the interfaces "natural" prefix, we
   will also need to advertise a static route to reach it.

  B.3) Don't advertise a gateway for any route

passt essentially proxy ARPs for the entire internet.

Advantages:
  * No need to allocate an address - in fact passt need not have any
    guest facing IP at all
  * Extends naturally if we ever have a guest<->passt transport that's
    point-to-point rather than pseudo-ethernet
Disadvantages:
  * Guest ARP / neighbour tables could get real big



The status quo is, roughly, A.1+B.1, except that we also enforce that
the host must have a default route, which sidesteps one of the
complications of B.1.  IIUC, this series is implementing A.2+B.1.

Thinking about it, I'm moderately convinced that B.1 is a bad idea.
I'm leaning towards B.2 - combining it with the NAT-to-host cleanups
to have a more concrete guest-visible address for passt itself - but
I'm also open to B.3.

I'm not sure about A.1 vs. A.2.  I was leaning towards A.2, but on
further consideration, I feel like the fact that A.1 automatically
works for routing changes on the host might outweigh the fact that he
guest only gets limited information (ICMP) about what's routable.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]