From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=none (p=none dis=none) header.from=gibson.dropbear.id.au Authentication-Results: passt.top; dkim=pass (2048-bit key; secure) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.a=rsa-sha256 header.s=202606 header.b=JzPykS8J; dkim-atps=neutral Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by passt.top (Postfix) with ESMTPS id 759BE5A026D for ; Thu, 02 Jul 2026 04:37:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202606; t=1782959865; bh=GF+xwopXqkEN9OM3qnIk2Nalh/Cc78NpCOsw2YwGXK0=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=JzPykS8JOfn5OI0e+7EWP4GVFPtqC1wslGUlDf2S63ISye2ZiiTBMYKj4JdqsIE7l TtBMNAQaud+QLvVWcfm+sM7D4FbDB3pAiLaCFbiezo9CWGGeTLdgTXbmSbg6G+f1gi IesMntd6aZ9xlu5WSQQUa+pbSdPUXB1EAAZL476bpn7VI7Cv2w4SjpWhncNvmN1fUW kjMLHW6/qa71FEOQe9iISEFGouFP7XYrwNBXdeERWXWD+FZnvGKo9ji7LFmdouXEHS k+0RDmk7QRfveSirbswjCbqmb2TR25f4aVs1hcAXT08udy/Ku61TA49/SJXPCItX4e +smiGAQU3nK3g== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4grLdj6bt5z58ly; Thu, 02 Jul 2026 12:37:45 +1000 (AEST) Date: Thu, 2 Jul 2026 12:27:27 +1000 From: David Gibson To: Laurent Vivier Subject: Re: [PATCH 2/8] tap: Make L4 sequence pools per-qpair for thread safety Message-ID: References: <20260616171052.3785909-1-lvivier@redhat.com> <20260616171052.3785909-3-lvivier@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="7Jj/m2fb0i90hPBa" Content-Disposition: inline In-Reply-To: <20260616171052.3785909-3-lvivier@redhat.com> Message-ID-Hash: UDROB6LPNH5PAHCLWMX55LU3R7IUHK7G X-Message-ID-Hash: UDROB6LPNH5PAHCLWMX55LU3R7IUHK7G X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --7Jj/m2fb0i90hPBa Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jun 16, 2026 at 07:10:46PM +0200, Laurent Vivier wrote: > The L4 sequence arrays tap4_l4[] and tap6_l4[] are used to batch > packets with the same L4 tuple within a single tap_handler() call. > They are global, but tap_handler() can be called concurrently from > different worker threads with different qpairs in vhost-user mode. >=20 > Make these arrays per-qpair by adding a VHOST_USER_MAX_VQS/2 first > dimension, indexed by the qpair parameter already available in > tap4_handler() and tap6_handler(). >=20 > Update tap_sock_update_pool() to initialize all qpair*seq entries. >=20 > Signed-off-by: Laurent Vivier Generally LGTM. Couple of nits and one broader design thought. > --- > tap.c | 32 +++++++++++++++++++------------- > 1 file changed, 19 insertions(+), 13 deletions(-) >=20 > diff --git a/tap.c b/tap.c > index 80912372e216..659df9d560d3 100644 > --- a/tap.c > +++ b/tap.c > @@ -606,7 +606,7 @@ static struct tap4_l4_t { > struct in_addr daddr; > =20 > struct pool_l4_t p; > -} tap4_l4[TAP_SEQS /* Arbitrary: TAP_MSGS in theory, so limit in users *= /]; > +} tap4_l4[VHOST_USER_MAX_VQS / 2][TAP_SEQS /* Arbitrary: TAP_MSGS in the= ory, so limit in users */]; Nit: VHOST_USER_MAX_VQS is being used often enough for sizing data structures that we might want a new #define for it. As in 1/8, I think an explicit cacheline alignment would be a good idea here too. > /** > * struct l4_seq6_t - Message sequence for one protocol handler call, IP= v6 > @@ -633,7 +633,7 @@ static struct tap6_l4_t { > uint8_t hop_limit; > =20 > struct pool_l4_t p; > -} tap6_l4[TAP_SEQS /* Arbitrary: TAP_MSGS in theory, so limit in users *= /]; > +} tap6_l4[VHOST_USER_MAX_VQS / 2][TAP_SEQS /* Arbitrary: TAP_MSGS in the= ory, so limit in users */]; Ditto. Also... maybe this would be more trouble for a first pass. But we're introducing a bunch of parallel per-qpair arrays. I'm wondering if it might be nicer to instead gather all the per-qpair structures together into a struct qpair_ctx or struct qpair_tap_ctx. Couple of advantages I can see (neither of which are must haves on their own): * Clearer to describe the locking semantics ("none, because it's thread private") for a single big structure than parallel slices of a whole bunch of other structures * Might give better memory locality for each thread > /** > * tap_packet_debug() - Print debug message for packet(s) from guest/tap > @@ -841,7 +841,7 @@ resume: > if (seq_count =3D=3D TAP_SEQS) > break; /* Resume after flushing if i < pool_tap4[qpair]->count */ > =20 > - for (seq =3D tap4_l4 + seq_count - 1; seq >=3D tap4_l4; seq--) { > + for (seq =3D tap4_l4[qpair] + seq_count - 1; seq >=3D tap4_l4[qpair]; = seq--) { > if (L4_MATCH(iph, uh, seq)) { > if (seq->p.count >=3D UIO_MAXIOV) > seq =3D NULL; > @@ -849,8 +849,8 @@ resume: > } > } > =20 > - if (!seq || seq < tap4_l4) { > - seq =3D tap4_l4 + seq_count++; > + if (!seq || seq < tap4_l4[qpair]) { > + seq =3D tap4_l4[qpair] + seq_count++; > L4_SET(iph, uh, seq); > pool_flush((struct pool *)&seq->p); > } > @@ -862,7 +862,7 @@ append: > packet_add((struct pool *)&seq->p, &data); > } > =20 > - for (j =3D 0, seq =3D tap4_l4; j < seq_count; j++, seq++) { > + for (j =3D 0, seq =3D tap4_l4[qpair]; j < seq_count; j++, seq++) { > const struct pool *p =3D (const struct pool *)&seq->p; > size_t k; > =20 > @@ -1089,7 +1089,7 @@ resume: > if (seq_count =3D=3D TAP_SEQS) > break; /* Resume after flushing if i < pool_tap6[qpair]->count */ > =20 > - for (seq =3D tap6_l4 + seq_count - 1; seq >=3D tap6_l4; seq--) { > + for (seq =3D tap6_l4[qpair] + seq_count - 1; seq >=3D tap6_l4[qpair]; = seq--) { > if (L4_MATCH(ip6h, proto, uh, seq)) { > if (seq->p.count >=3D UIO_MAXIOV) > seq =3D NULL; > @@ -1097,8 +1097,8 @@ resume: > } > } > =20 > - if (!seq || seq < tap6_l4) { > - seq =3D tap6_l4 + seq_count++; > + if (!seq || seq < tap6_l4[qpair]) { > + seq =3D tap6_l4[qpair] + seq_count++; > L4_SET(ip6h, proto, uh, seq); > pool_flush((struct pool *)&seq->p); > } > @@ -1110,7 +1110,7 @@ append: > packet_add((struct pool *)&seq->p, &data); > } > =20 > - for (j =3D 0, seq =3D tap6_l4; j < seq_count; j++, seq++) { > + for (j =3D 0, seq =3D tap6_l4[qpair]; j < seq_count; j++, seq++) { > const struct pool *p =3D (const struct pool *)&seq->p; > size_t k; > =20 > @@ -1607,9 +1607,15 @@ static void tap_sock_update_pool(void *base, size_= t size) > pool_tap6[i] =3D (struct pool *)&pool_tap6_storage[i]; > } > =20 > - for (i =3D 0; i < TAP_SEQS; i++) { > - tap4_l4[i].p =3D PACKET_INIT(pool_l4, UIO_MAXIOV, base, size); > - tap6_l4[i].p =3D PACKET_INIT(pool_l4, UIO_MAXIOV, base, size); > + for (i =3D 0; i < VHOST_USER_MAX_VQS / 2; i++) { > + unsigned int j; > + > + for (j =3D 0; j < TAP_SEQS; j++) { > + tap4_l4[i][j].p =3D PACKET_INIT(pool_l4, UIO_MAXIOV, > + base, size); > + tap6_l4[i][j].p =3D PACKET_INIT(pool_l4, UIO_MAXIOV, > + base, size); > + } > } > } > =20 > --=20 > 2.54.0 >=20 --=20 David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson --7Jj/m2fb0i90hPBa Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEO+dNsU4E3yXUXRK2zQJF27ox2GcFAmpFzIoACgkQzQJF27ox 2Gf7iQ//S68TAklMrRBZkjCqgDLizpryyL/RizhLN1D7CQSQhNFKRMYu6VjWYjDt KsTAsMEKc/vZly3gjTXRMrX7jfuLaUbFFX/pRmoHR6wURsN3+CEKUMBEMepKU/zf ODQ4+kAklsZAM8i62AhkUUHWrwH1hU8pUiEY955tMn0xYVHfidiq9MbocP/7+daS N+kSASKtIkRjav7FoJbqEtM9UjgF+8L4ckVLI4OOUokha7aKHQqrfioeSbBcxt6R yUDCbg02x/VxTShajtBM8DCsnZDn+HuvUzAtX2JTmJD7ZHV05mSAFHgoLbJR2j4S qdZ+TXRxs6Ccco4gnU2SV9+J38f/0UFda50KDDCBtFZg9GDkrLw2RDxkr44l/R37 NjfFHwCduxPXfZKmmDNe31TGrjKxCK9aLyMOxpurEw8I6xgIP82eGkcIQT4AQqni dSqKNT51oQYje/gTV5woR59HjMp7MmO40P61a4qchP06vpW9iixdHZJIC4vafuf8 QZk0BbkHCA/gMdx4aTKAQOU6X2a0lO4P7Bh2OjSplVmkJIEzTBed/Vp/tug0s9H4 Mz+2oye6ea2vtGdb5gArWSrQX08nOhFVkHRUXW+W8uOkC343BvIeQK8m2SYNQlsR J78F7mJNjJVf5oSRVTEfwBuDq+c2elLcsvf3A6rPVhWzZsC9TzE= =1c/a -----END PGP SIGNATURE----- --7Jj/m2fb0i90hPBa--