From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by passt.top (Postfix) with ESMTPS id 235EC5A026F for ; Wed, 6 Mar 2024 06:09:31 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202312; t=1709701768; bh=OAd7CrZzIoSO2QCjREHHBgvYTeLvlLMP7iMdEvN0KsI=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=RSGBpzgz1EUgiUgCPyMcYtGDOYUU80aXUPjNAzuv3njJt1VrfSBMhDJaK4bxw/OZL K5z2uzCffbsbpmnZ1ivEIlicFft91ngT7fIYq5oiWt0dBSZ5ltDBY3ldPtKhDc+siB 3NfD2Nwk7SvSSSFJN7QM4V1iWCO/1rX2dcKWTT9F8bixtz3+p0yv+CICe7zDVricVH b3tI0tycvPDnB20YDWhraiOxepM+UMUk8KO4i7f5FS10+b7HvgHy3nl/EiUNOqROEp UQdscZTiynq8BO7P+TZLvEjYoajSHnZgPVC8sym2a8kGGijG6yuPlNzpEm+e+o2tgd 7lO5wAP3Dyhnw== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4TqL8828wPz4wcD; Wed, 6 Mar 2024 16:09:28 +1100 (AEDT) Date: Wed, 6 Mar 2024 16:09:23 +1100 From: David Gibson To: Stefano Brivio Subject: Re: [PATCH v3 7/9] checksum: introduce functions to compute the header part checksum for TCP/UDP Message-ID: References: <20240229080509.4f534831@elisabeth> <20240229095625.557367ab@elisabeth> <20240229151553.60d5cf18@elisabeth> <20240301075651.42ec7145@elisabeth> <20240304120040.1cebc230@elisabeth> <20240304234717.5f697efd@elisabeth> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="dvz+zmFoP9JZfy4V" Content-Disposition: inline In-Reply-To: <20240304234717.5f697efd@elisabeth> Message-ID-Hash: D5YT3LHZMGZBOIURF3CZIHKEUXKJYXWI X-Message-ID-Hash: D5YT3LHZMGZBOIURF3CZIHKEUXKJYXWI X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Laurent Vivier , passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --dvz+zmFoP9JZfy4V Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Mar 04, 2024 at 11:47:17PM +0100, Stefano Brivio wrote: > On Mon, 4 Mar 2024 12:00:40 +0100 > Stefano Brivio wrote: >=20 > > On Mon, 4 Mar 2024 12:54:12 +1100 > > David Gibson wrote: > >=20 > > > On Fri, Mar 01, 2024 at 07:56:51AM +0100, Stefano Brivio wrote: =20 > > > > On Fri, 1 Mar 2024 10:09:39 +1100 > > > > David Gibson wrote: > > > > =20 > > > > > On Thu, Feb 29, 2024 at 03:15:53PM +0100, Stefano Brivio wrote: = =20 > > > > > > On Thu, 29 Feb 2024 09:56:25 +0100 > > > > > > Stefano Brivio wrote: > > > > > > =20 > > > > > > > On Thu, 29 Feb 2024 19:49:09 +1100 > > > > > > > David Gibson wrote: > > > > > > > =20 > > > > > > > > On Thu, Feb 29, 2024 at 08:05:09AM +0100, Stefano Brivio wr= ote: =20 > > > > > > > > > On Thu, 29 Feb 2024 11:38:53 +1100 > > > > > > > > > David Gibson wrote: > > > > > > > > > =20 > > > > > > > > > > On Wed, Feb 28, 2024 at 02:26:18PM +0100, Laurent Vivie= r wrote: =20 > > > > > > > > > > > On 2/19/24 04:08, David Gibson wrote: =20 > > > > > > > > > > > > On Sat, Feb 17, 2024 at 04:07:23PM +0100, Laurent V= ivier wrote: =20 > > > > > > > > > > > > > > > > > > > > > > > > [...] > > > > > > > > > > > > =20 > > > > > > > > > > > > > +/** > > > > > > > > > > > > > + * proto_ipv6_header_psum() - Calculates the par= tial checksum of an > > > > > > > > > > > > > + * IPv6 header for UDP or TCP > > > > > > > > > > > > > + * @payload_len: Payload length > > > > > > > > > > > > > + * @proto: Protocol number > > > > > > > > > > > > > + * @saddr: Source address > > > > > > > > > > > > > + * @daddr: Destination address > > > > > > > > > > > > > + * Returns: Partial checksum of the IPv6 header > > > > > > > > > > > > > + */ > > > > > > > > > > > > > +uint32_t proto_ipv6_header_psum(uint16_t payload= _len, uint8_t protocol, > > > > > > > > > > > > > + struct in6_addr saddr, struct in6_addr daddr= ) =20 > > > > > > > > > > > >=20 > > > > > > > > > > > > Hrm, this is passing 2 16-byte IPv6 addresses by va= lue, which might > > > > > > > > > > > > not be what we want. =20 > > > > > > > > > > >=20 > > > > > > > > > > > The idea here is to avoid the pointer alignment probl= em (&ip6h->saddr and > > > > > > > > > > > &ip6h->daddr can be misaligned). =20 > > > > > > > > > >=20 > > > > > > > > > > Ah, right. That's a neat idea, but I'm not sure it rea= lly helps: I > > > > > > > > > > think it will just move the misaligned access from insi= de the function > > > > > > > > > > to the call site, where we try to marshal the parameter= from something > > > > > > > > > > unaligned. =20 > > > > > > > > >=20 > > > > > > > > > I haven't tested this yet, but note that this is generall= y okay: the > > > > > > > > > problem is *dereferencing* an unaligned pointer. But if y= ou load memory > > > > > > > > > from an aligned pointer, and extract a value from this me= mory, it's all > > > > > > > > > fine. =20 > > > > > > > >=20 > > > > > > > > Right, that's kind of what I'm getting at. Assuming this v= alue starts > > > > > > > > in an unaligned buffer, then in order to pass this by value= the caller > > > > > > > > will need to load from that unaligned pointer. AFAIK, the = compiler > > > > > > > > will base the type of loads only on the pointed to type, wh= ich isn't > > > > > > > > changed whether we dereference in the caller or the callee. > > > > > > > > =20 > > > > > > > > >=20 > > > > > > > > > Speaking MIPS, this is not safe on all CPU models: > > > > > > > > >=20 > > > > > > > > > la $1, 1002 # s1 now contains the value 1002 > > > > > > > > > lw $2, 0($1) # load word from memory at 1002 + 0 into s2 > > > > > > > > >=20 > > > > > > > > > but this is: > > > > > > > > >=20 > > > > > > > > > la $1, 1000 # s1 now contains the value 1000 > > > > > > > > > la $2, 1004 # s3 now contains the value 1004 > > > > > > > > > lw $3, 0($1) # load word from memory at 1000 + 0 into s3 > > > > > > > > > lw $4, 0($3) # load word from memory at 1004 + 0 into s4 > > > > > > > > > sll $5, $3, 16 # 16-bit shift left s3 into s5 > > > > > > > > > srl $6, $4, 16 # 16-bit shift right s4 into s6 > > > > > > > > > or $2, $5, $6 # OR s5 and s6 into s2 =20 > > > > > > > >=20 > > > > > > > > Right, but I don't think merely moving the dereference to t= he caller > > > > > > > > will necessarily induce the compiler to generate this rathe= r than the > > > > > > > > former. =20 > > > > > > >=20 > > > > > > > Oh, oops, I didn't realise this was the case (I haven't revie= wed the > > > > > > > patch yet). =20 > > > > > >=20 > > > > > > ...no, that's not the case. Dereferencing 'iph' from > > > > > > struct tcp[46]_l2_buf_t is fine: > > > > > >=20 > > > > > > struct tcp4_l2_buf_t { > > > > > > uint8_t pad[2]; /* = 0 2 */ > > > > > > struct tap_hdr taph; /* = 2 18 */ > > > > > > struct iphdr iph; /* = 20 20 */ > > > > > > [...] > > > > > > } __attribute__((__packed__)); > > > > > >=20 > > > > > > struct tcp6_l2_buf_t { > > > > > > uint8_t pad[2]; /* = 0 2 */ > > > > > > struct tap_hdr taph; /* = 2 18 */ > > > > > > struct ipv6hdr ip6h; /* = 20 40 */ > > > > > > [...] > > > > > > } __attribute__((__packed__)); > > > > > >=20 > > > > > > The problematic structures are the UDP buffers: > > > > > >=20 > > > > > > struct udp4_l2_buf_t { > > > > > > struct sockaddr_in s_in; /* = 0 16 */ > > > > > > struct tap_hdr taph; /* = 16 18 */ > > > > > > struct iphdr iph; /* = 34 20 */ > > > > > > [...] > > > > > > } __attribute__((__aligned__(4))); > > > > > >=20 > > > > > > and for UDP, this patch is dereferencing buffer pointers only, = not > > > > > > pointers to headers. =20 > > > > >=20 > > > > > Ok... but my point remains, I'm not seeing that passing the addre= ss by > > > > > value actually helps - it just seems to change whether we need to > > > > > handle the unaligned load in the caller or the callee. =20 > > > >=20 > > > > For UDP and IPv4 (from 6/9): > > > >=20 > > > > + b->iph.check =3D csum_ip4_header(b->iph.tot_len, IPPROTO_UD= P, > > > > + b->iph.saddr, b->iph.daddr); > > > >=20 > > > > and for IPv6 (this patch): > > > >=20 > > > > + b->uh.check =3D csum(&b->uh, ntohs(b->ip6h.payload_len), > > > > + proto_ipv6_header_psum(b->ip6h.payload_l= en, > > > > + IPPROTO_UDP, > > > > + b->ip6h.saddr, > > > > + b->ip6h.daddr)); > > > >=20 > > > > these cause loads starting from 'b', which is aligned, instead of > > > > passing 'iph' or 'ip6h', unaligned, and loading from there. =20 > > >=20 > > > No... the loads are still from b->ip6h.saddr, b->ip6h.daddr and > > > b->ip6h.payload_len. =20 > >=20 > > It depends how we define "loading from" -- the problem, in general, is > > not the memory location per se, the problem is dereferencing memory > > pointers. > >=20 > > I plan to try an example on MIPS in a bit [...] >=20 > Actually, armhf first (for clarity): >=20 > $ cat align.c > #include > #include >=20 > struct disarray { > uint8_t oops; > uint32_t v1; > uint32_t v2; > } __attribute__((packed, aligned(__alignof__(unsigned int)))); >=20 > void f1(uint32_t *v1) { > *v1 +=3D 42; > } >=20 > uint32_t f2(uint32_t v2) { > return v2++; > } >=20 > int main() > { > struct disarray d =3D { 0x55, 0xaa, 0xaa }; >=20 > f1(&d.v1); > f2(d.v2); >=20 > fprintf(stdout, "%08x %08x", d.v1, d.v2); > } >=20 > $ arm-linux-gnueabihf-gcc-12 -g -O0 -fno-stack-protector -fomit-frame-poi= nter -mno-unaligned-access -o align align.c > align.c: In function =E2=80=98main=E2=80=99: > align.c:22:8: warning: taking address of packed member of =E2=80=98struct= disarray=E2=80=99 may result in an unaligned pointer value [-Waddress-of-p= acked-member] > 22 | f1(&d.v1); > | ^~~~~ >=20 > $ arm-linux-gnueabihf-objdump -S --disassemble=3Dmain align > [...] > f1(&d.v1); > 562: ab01 add r3, sp, #4 > 564: 3301 adds r3, #1 > 566: 4618 mov r0, r3 > 568: f7ff ffde bl 528 > [...] >=20 > before the call to f1(), the address in r3 is not aligned (we just > added #1), despite -mno-unaligned-access. I guess gcc can only warn > about that, but not fix it. >=20 > This: > https://gcc.gnu.org/onlinedocs/gcc/ARM-Options.html >=20 > says: > -munaligned-access > -mno-unaligned-access >=20 > Enables (or disables) reading and writing of 16- and 32- bit values f= rom addresses that are not 16- or 32- bit aligned. By default unaligned acc= ess is disabled for all pre-ARMv6, all ARMv6-M and for ARMv8-M Baseline arc= hitectures, and enabled for all other architectures. If unaligned access is= not enabled then words in packed data structures are accessed a byte at a = time.=20 >=20 > Implying, I guess, that on those architectures unaligned accesses > shouldn't be done. I think Thumb mode also has issues with this, by > the way.=20 >=20 > And in f1() we just have a ldr from that address (passed on r0): > void f1(uint32_t *v1) { > 528: b082 sub sp, #8 > 52a: 9001 str r0, [sp, #4] > *v1 +=3D 42; > 52c: 9b01 ldr r3, [sp, #4] > 52e: 681b ldr r3, [r3, #0] > 530: f103 022a add.w r2, r3, #42 @ 0x2a >=20 > $ arm-linux-gnueabihf-objdump -S --disassemble=3Df1 align > [...] > *v1 +=3D 42; > 52c: 9b01 ldr r3, [sp, #4] > 52e: 681b ldr r3, [r3, #0] > 530: f103 022a add.w r2, r3, #42 @ 0x2a >=20 > ...but the call to f2() is fine: we load with offset 8 from the stack > pointer, shift word right, load from offset 12, shift word left, OR: >=20 > $ arm-linux-gnueabihf-objdump -S --disassemble=3Dmain align > [...] > f2(d.v2); > 56c: 9b02 ldr r3, [sp, #8] > 56e: 0a1b lsrs r3, r3, #8 > 570: f89d 200c ldrb.w r2, [sp, #12] > 574: 0612 lsls r2, r2, #24 > 576: 4313 orrs r3, r2 > 578: 4618 mov r0, r3 > 57a: f7ff ffe0 bl 53e > [...] Huh. Ok, so I guess the compiler realises it's doing a load from a packed structure and generates the necessary fixup code. I thought it would only consider the type of the actually loaded value. Are you sure it still does this correctly when optimization is enabled? >=20 > Now on to MIPS (MIPS32): >=20 > $ mips-linux-gnu-gcc-12 -g -O0 -fno-stack-protector -fomit-frame-pointer = -mno-unaligned-access -o align align.c > align.c: In function =E2=80=98main=E2=80=99: > align.c:22:8: warning: taking address of packed member of =E2=80=98struct= disarray=E2=80=99 may result in an unaligned pointer value [-Waddress-of-p= acked-member] > 22 | f1(&d.v1); > | ^~~~~ >=20 > $ mips-linux-gnu-objdump -S --disassemble=3Dmain align > [...] > f1(&d.v1); > 7bc: 27a20019 addiu v0,sp,25 > 7c0: 00402025 move a0,v0 > 7c4: 8f82802c lw v0,-32724(gp) > 7c8: 0040c825 move t9,v0 > 7cc: 0411ffe0 bal 750 > 7d0: 00000000 nop > 7d4: 8fbc0010 lw gp,16(sp) > [...] >=20 > '&d.v1' is passed in a0, again unaligned (stack pointer plus 25). And f1() > uses it just like that: >=20 > $ mips-linux-gnu-objdump -S --disassemble=3Df1 align > [...] > void f1(uint32_t *v1) { > 750: afa40000 sw a0,0(sp) > *v1 +=3D 42; > 754: 8fa20000 lw v0,0(sp) > 758: 8c420000 lw v0,0(v0) > 75c: 2443002a addiu v1,v0,42 > [...] >=20 > while the call to f2() is, again, fine: >=20 > $ mips-linux-gnu-objdump -S --disassemble=3Dmain align > f2(d.v2); > 7e0: 8ba2001d lwl v0,29(sp) > 7e4: 9ba20020 lwr v0,32(sp) > 7e8: 00402025 move a0,v0 > 7ec: 8f828030 lw v0,-32720(gp) > 7f0: 0040c825 move t9,v0 > 7f4: 0411ffdf bal 774 > 7f8: 00000000 nop > 7fc: 8fbc0010 lw gp,16(sp) >=20 > two loads, from stack pointer + 29 and stack pointer + 32. MIPS32 has lwl > and lwr (the infamous US4814976A patent, now expired) to avoid load plus > shift plus OR. >=20 > Now, you might argue that what I'm describing here might simply be gcc's > behaviour, and if gcc avoids unaligned loads as long as we don't pass > unaligned pointers around, that's not any better for us -- other compilers > might do things differently. >=20 > And... yes, packed structures are actually a GNU extension: C standards > don't say anything about loads like my f1(d.v2) call above, so all I'm > showing here is that a particular compiler is fine with these accesses, > but not unaligned pointers. >=20 > On the other hand, this seems to be a well established behaviour, and I > don't think we could realistically drop every load of unaligned *values*. > Unaligned pointers, we currently don't dereference any, because gcc warns > otherwise. >=20 > So, practically speaking, I guess as long as we avoid dereferencing > unaligned pointers, we should be fine? >=20 --=20 David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson --dvz+zmFoP9JZfy4V Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEO+dNsU4E3yXUXRK2zQJF27ox2GcFAmXn+noACgkQzQJF27ox 2GeSqQ/+Pj5cXeKsbiNoMfPnp5q9U0IS524crskYY6A0U//mqPNTx/4CViTkCIvs T63mR5gY/Tl17lRBzThHMKMnBM5XjODIWBFFPRiGc2QI+urzPbKSpVhS9EX76tOc rUUdAaHo6G/iyQ8xit4y1naKkB1ZZ2JrKJZ8bk3UNyI8HNpSlHts54brMNdSO6wt MyMun2uQRZfDGswBpgOK3blheaB+MVlelCbidLPx/FKP2EuXEt5Q1QkpopXvrSk3 i8p4PW1ls3CR4NjKc5coyHmLebQVSs2VxRSX3cPmldUANWOaUbEm79XbBmPbYcH1 uX1/4Jqvq9b4hfrSkNvLFEN1YfCCJMh4M8+cUARCvSTcLIg5DcJo3RyQOfxEGUOP wU1SI/5lTJPsHgixrqmvyX0Jv50nmlioJpBfJoimELZIvHed1/7knVWLTQquzXje R00cuLGXQ8CYQ+u1OAHd2+nldMu4EtH4OMM+C1dGvEvSyIte2D3GQspwXl4gQi4a jYsqujGx04eNbhuAYTfAS4YQyg4RpJ6pa8mVkR6eI0yfM+QGv64MT8nMb9aZUdFp A4D3wycfwA2hU8j5tEantN8aJ0klhqibdS+xwpXNC7GFfmPGhW28s6/10MNNpW5d fUAMjmmaZ3PINovWn1SMxtsxw06C8m+6JPsCFgiLmeRxGSsxRXA= =76Vy -----END PGP SIGNATURE----- --dvz+zmFoP9JZfy4V--