From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=none (p=none dis=none) header.from=gibson.dropbear.id.au Authentication-Results: passt.top; dkim=pass (2048-bit key; secure) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.a=rsa-sha256 header.s=202504 header.b=roasNxMR; dkim-atps=neutral Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by passt.top (Postfix) with ESMTPS id A253D5A0272 for ; Tue, 15 Apr 2025 09:16:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202504; t=1744701386; bh=v0JdihLTOG0OdMw/Sm5ANn663FrozhkY/swL0vOiQmg=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=roasNxMRIj6vedX39VrZJO2m35K+wsmnNXiJCYorEDZThTt+98z0eCOfBeSLX9xPv krcgijIDaXVi7EmTwIRaJDuP/aNI0vEs2NSEYWwtzsfi05RM9iwJxiFSMd7AtO2Trp JGGqa230l5DWkpT+78papujNCe6fwnC00Ng70fxUMGoMuahiZ2hJZ7xMyHLjek/U36 bn0lxyhp3bnwUT5YK/5lFlIBrwO4R3uF5kjA6rGThGfkOu1Gwm4IIgsoyEmwX7ISm5 jay9gaPeKvvybFs/Rklvy8kmuq5RMJw8lHhg6Ky38IW3ncXH9V71ioxpK0rh92BvT+ I3GFWiAJqurnA== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4ZcFnk4svWz4xG0; Tue, 15 Apr 2025 17:16:26 +1000 (AEST) Date: Tue, 15 Apr 2025 15:28:51 +1000 From: David Gibson To: Stefano Brivio Subject: Re: [PATCH] udp: Fix breakage of UDP error handling by PKTINFO support Message-ID: References: <20250414035853.1379261-1-david@gibson.dropbear.id.au> <20250414115641.03064d86@elisabeth> <20250414191251.3623406b@elisabeth> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="/6jYDzHOn6W3waIx" Content-Disposition: inline In-Reply-To: <20250414191251.3623406b@elisabeth> Message-ID-Hash: VL76DCUIQDLFLWW5CU2EWXTWRDBPDIE7 X-Message-ID-Hash: VL76DCUIQDLFLWW5CU2EWXTWRDBPDIE7 X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top, Jon Maloy X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --/6jYDzHOn6W3waIx Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Apr 14, 2025 at 07:12:51PM +0200, Stefano Brivio wrote: > A couple of further observations: >=20 > On Mon, 14 Apr 2025 11:56:41 +0200 > Stefano Brivio wrote: >=20 > > On Mon, 14 Apr 2025 13:58:53 +1000 > > David Gibson wrote: > >=20 > > > We recently enabled the IP_PKTINFO / IPV6_RECVPKTINFO socket options = on our > > > UDP sockets. This lets us obtain and properly handle the specific lo= cal > > > address used when we're "listening" with a socket on 0.0.0.0 or ::. > > >=20 > > > However, the PKTINFO cmsgs this option generates appear on error queue > > > messages as well as regular datagrams. udp_sock_recverr() doesn't ex= pect > > > this and so flags an unrecoverable error when it can't parse the cont= rol > > > message. > > >=20 > > > Correct this by adding space in udp_sock_recverr()s control buffer fo= r the > > > additional PKTINFO data, and scan through all cmsgs for the RECVERR, = rather > > > than only looking at the first one. > > >=20 > > > Link: https://bugs.passt.top/show_bug.cgi?id=3D99 > > > Fixes: f4b0dd8b06 ("udp: Use PKTINFO cmsgs to get destination address= =2E.") > > > Reported-by: Stefano Brivio > > >=20 > > > Signed-off-by: David Gibson =20 > >=20 > > The patch looks good to me, but I'm hitting something in my tests (with > > my recent DNS fix) that's perhaps not intended. > >=20 > > On the host: > >=20 > > $ cat /etc/resolv.conf > > nameserver 127.0.0.1 > >=20 > > ...and nobody is listening on that address. Podman passes > > --dns-forward 169.254.1.1 (default) and in the container: > >=20 > > $ podman run --net=3Dpasta:-d,-l,/tmp/pasta.log --rm -ti alpine sh > >=20 > > I do: > >=20 > > / # nslookup google.com 169.254.1.1 > > ;; connection timed out; no servers could be reached > >=20 > > which is expected. But logs show a warning: > >=20 > > 49.5377: Flow 0 (NEW): FREE -> NEW > > 49.5377: Flow 0 (INI): NEW -> INI > > 49.5377: Flow 0 (INI): TAP [88.198.0.164]:43458 -> [169.254.1.= 1]:53 =3D> ? > > 49.5377: Flow 0 (TGT): INI -> TGT > > 49.5377: Flow 0 (TGT): TAP [88.198.0.164]:43458 -> [169.254.1.= 1]:53 =3D> HOST [0.0.0.0]:43458 -> [127.0.0.1]:53 > > 49.5377: Flow 0 (UDP flow): TGT -> TYPED > > 49.5377: Flow 0 (UDP flow): TAP [88.198.0.164]:43458 -> [169.2= 54.1.1]:53 =3D> HOST [0.0.0.0]:43458 -> [127.0.0.1]:53 > > 49.5378: Flow 0 (UDP flow): Side 0 hash table insert: bucket: = 148325 > > 49.5378: Flow 0 (UDP flow): Side 1 hash table insert: bucket: = 309967 > > 49.5378: Flow 0 (UDP flow): TYPED -> ACTIVE > > 49.5378: Flow 0 (UDP flow): TAP [88.198.0.164]:43458 -> [169.2= 54.1.1]:53 =3D> HOST [127.0.0.1]:43458 -> [127.0.0.1]:53 > > 49.5378: WARNING: Error peeking at socket address: Connection refused > > 49.5379: ICMP error on UDP socket 208: Connection refused > > 49.5379: ICMP error on UDP socket 208: Connection refused > > 52.0404: ICMP error on UDP socket 208: Connection refused > > 52.0404: ICMP error on UDP socket 208: Connection refused > >=20 > > and I'm not sure if that warning is intended. >=20 > The warning actually comes from the previous series, not from "[PATCH > 0/3] Properly preseve local addresses for UDP flows", but rather from > commit 84ab1305faba ("udp: Polish udp_vu_sock_info() and remove from > vu specific code"). Right, I was beginning to suspect that from earlier information. My own investigations today showed that as a problem, and I have a fix for that along with some related issues. > I couldn't figure out why we get that error *there* but I didn't try > hard. If it's unavoidable, I guess we should change that to a debug() > message, otherwise a guest / container doing a couple of UDP port scans > could flood host logs. Yes, they definitely need to be trace level messages. I know who it's happening there, details in my upcoming commit messages. > > By the way, I have the > > feeling that it now takes longer (with the whole IP_PKTINFO thing) for > > nslookup to fail, as if those ICMP errors were not relayed anymore, but > > I'm not sure about this, and I didn't investigate yet. >=20 > I couldn't reproduce the "fast" nslookup failure I'm fairly sure I had > seen at some point. I tried reverting my fixes, and both of your series > separately, but I can't get that to work. So. I think there are two different problems stopping this from working. 1) The new udp_flush_flow() is essentially discarding the error without it passing through the EPOLLERR path and triggering the correct ICMP. 2) If the host side error is generated locally (address 127.0.0.1), we send that address untranslated to the guest where it makes no sense. > I'm attaching a packet capture of the current behaviour. All the > relevant details are the same as without your two series (and with or > without my fix). >=20 > I'm not sure if we should map the source address of the ICMP messages > (Jon, thoughts?) instead of leaving it as 127.0.0.1, in case > fwd_nat_from_tap() changed the outbound source address. >=20 > That is, if we have: >=20 > 4.2401: Flow 0 (UDP flow): TAP [88.198.0.164]:34843 -> [169.254.= 1.1]:53 =3D> HOST [0.0.0.0]:34843 -> [127.0.0.1]:53 > 4.2402: ICMP error on UDP socket 209: Connection refused >=20 > we currently send an ICMP message from 127.0.0.1 (I guess ignored) Yes, that will certainly be ignored. > instead of using 169.254.1.1 as source. Should udp_send_tap_icmp4() > call tap_icmp4_send() with 'fromside->oaddr' as source (instead of > 'saddr'), where 'fromside' is the flow side opposite to 'toside'? No. fromside->oaddr will likely be the same address as the guest has, so (depending how paranoid it's configuration is), it's still likely to discard it. We need to do the reverse translation of --map-guest-addr and --map-host-loopback; this is the first time we've needed such translation on an address that's not one of the main flow addresses. We didn't think of it earlier, but it makes sense: the ICMP can come from some intermediate address that's not either endpoint of the flow, so we need to NAT it specifically. > I'm not entirely sure if those ICMP messages are ignored, though. > Maybe it's all fine the way it is and it's just ports that need to > match, and it's normal that nslookup retries after a while... I'm pretty sure they are. I recently encountered a similar thing for a different reason (I was testing an older version without 65cca54b ("udp: correct source address for ICMP messages"). --=20 David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson --/6jYDzHOn6W3waIx Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEO+dNsU4E3yXUXRK2zQJF27ox2GcFAmf97oQACgkQzQJF27ox 2Ge/AQ/+JJYhSY8KYyG4xACY/O/eEsWfEz2bkhW+sXiSOLKSDnx+9SFpjQ9iPFUH 5W/blQSnaVL24xiCu9WnflilCl/ySinjsrcRaYziVz5arTp08pWCI2E3nawSkskr wBzIVHzxNVg1tPGr9k70W/wL7ZocGrgTyA1d25qKTXcTmMmIG3mOHTXVtcuppxW2 ivF1tMtnC6b5GR/Ua563z5rss/Np1tY3hGr2t6QoDBLw02/jhxp9oiGhnDmqXizs o7xROF+3dQZ2AQX1zVwn4L0Xl4YJ7PYKbNuXeTya5ZtAGQ/MEECN6+By21Ay6Wjf SK/hWKqrOnxdpABxidC+vzAO798UOqSHY7y7QLoUoN+CZzeGqHn2yeuVmSX5iwKr GZ7tGW8or3/B7BtFWSRrCIvvCFWnGqCiFzt6Ag2y4+xdA4Qt4PV/7Ksc2Mwc8Bqh uc9HR2pu0Pl0j9LPQSNO9v2RoJU09iAIa5G3tn0AUDTR3IQB3lu8JrZWNp3ySOaA 9wQB3Sf0P5SDCwXIFZG4MoWeTL9H1BgrNq9kyuaqEC6fazi+w36BBc0xMA1NgJdw GeP/EGcL+PIAsa7diCnMmpO/JcrvlOo6AHJEtClFN3WvEKMniYDdpPgrXZjhHSF2 vr3bjXeooaCdyIHTA0Gj8whY392hScStLqWbYqcyobyhHh1fp0s= =cH1F -----END PGP SIGNATURE----- --/6jYDzHOn6W3waIx--