From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=none (p=none dis=none) header.from=gibson.dropbear.id.au Authentication-Results: passt.top; dkim=pass (2048-bit key; secure) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.a=rsa-sha256 header.s=202602 header.b=gLNGc9uL; dkim-atps=neutral Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by passt.top (Postfix) with ESMTPS id 42D725A0265 for ; Thu, 21 May 2026 08:00:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202602; t=1779343247; bh=mS9NkLHqK0P/U9r+i4yLYKDjuc/pOZLd7CnX4ws878Q=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=gLNGc9uLOE11q0bI47BWoeSbfk+AjU/yFQDCEHSW21p7R8ScNvhMkKvugQbjaPTt+ OHAlSxDYp8+fNVnk4le1Qh9adLs6NoODwquYY/XJrrO5CYiXqK7pTzks890Byu6kGH 7Z0n4eYRtCszmMbQxvmgW7wkiQQWi7PULyZY7OsH4tf+5yh8ecSOMEHKCRJ3gBlVRd Hm0eR7Gtt69F9QTJT0vKDy5hVpSzP3rjjawcp/b6YeAuELiy7TEuxddz1nvtocXeQj jBatfQ33eaivDY5KZHKNIb8Dp3qGOsOo7ZOUHmg1B5XpoxfX/41HkQvGmJZVZEGUNr oAbuy49CCxflQ== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4gLd7M4qGCz4wKy; Thu, 21 May 2026 16:00:47 +1000 (AEST) Date: Thu, 21 May 2026 16:00:21 +1000 From: David Gibson To: Laurent Vivier Subject: Re: [PATCH v6 3/4] tcp_vu: Support multibuffer frames in tcp_vu_sock_recv() Message-ID: References: <20260416161618.3826904-1-lvivier@redhat.com> <20260416161618.3826904-4-lvivier@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="C2imQKwC+qGxr/Ij" Content-Disposition: inline In-Reply-To: Message-ID-Hash: 3MKD4JA475EAU4AER4Q6374WIQEETH3B X-Message-ID-Hash: 3MKD4JA475EAU4AER4Q6374WIQEETH3B X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --C2imQKwC+qGxr/Ij Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, May 20, 2026 at 03:46:48PM +0200, Laurent Vivier wrote: > On 5/11/26 10:24, David Gibson wrote: > > On Thu, Apr 16, 2026 at 06:16:17PM +0200, Laurent Vivier wrote: > > > Previously, tcp_vu_sock_recv() assumed a 1:1 mapping between virtqueue > > > elements and iovecs (one iovec per element), enforced by an ASSERT. > > > This prevented the use of virtqueue elements with multiple buffers > > > (e.g. when mergeable rx buffers are not negotiated and headers are > > > provided in a separate buffer). > > >=20 > > > Introduce a struct vu_frame to track per-frame metadata: the range of > > > elements and iovecs that make up each frame, and the frame's total si= ze. > > > This replaces the head[] array which only tracked element indices. > > >=20 > > > A separate iov_msg[] array is built for recvmsg() by cloning the data > > > portions (after stripping headers) using iov_tail helpers. > > >=20 > > > Then a frame truncation after recvmsg() properly walks the frame and > > > element arrays to adjust iovec counts and element counts. > > >=20 > > > Signed-off-by: Laurent Vivier > >=20 > > LGTM apart from a couple of minor points noted below. > >=20 > > > --- > > > tcp_vu.c | 174 ++++++++++++++++++++++++++++++++++++----------------= --- > > > 1 file changed, 113 insertions(+), 61 deletions(-) > > >=20 > > > diff --git a/tcp_vu.c b/tcp_vu.c > > > index 2017aec90342..96b16007701d 100644 > > > --- a/tcp_vu.c > > > +++ b/tcp_vu.c > > > @@ -35,9 +35,24 @@ > > > #include "vu_common.h" > > > #include > > > -static struct iovec iov_vu[VIRTQUEUE_MAX_SIZE + DISCARD_IOV_NUM]; > > > +static struct iovec iov_vu[VIRTQUEUE_MAX_SIZE]; > > > static struct vu_virtq_element elem[VIRTQUEUE_MAX_SIZE]; > > > -static int head[VIRTQUEUE_MAX_SIZE + 1]; > > > + > > > +/** > > > + * struct vu_frame - Descriptor for a TCP frame mapped to virtqueue = elements > > > + * @idx_element: Index of first element in elem[] for this frame > > > + * @num_element: Number of virtqueue elements used by this frame > > > + * @idx_iovec: Index of first iovec in iov_vu[] for this frame > > > + * @num_iovec: Number of iovecs covering this frame's buffers > > > + * @size: Total frame size including all headers > > > + */ > > > +static struct vu_frame { > > > + int idx_element; > > > + int num_element; > > > + int idx_iovec; > > > + int num_iovec; > > > + size_t size; > > > +} frame[VIRTQUEUE_MAX_SIZE]; > > > /** > > > * tcp_vu_hdrlen() - Sum size of all headers, from TCP to virtio-net > > > @@ -174,8 +189,8 @@ int tcp_vu_send_flag(const struct ctx *c, struct = tcp_tap_conn *conn, int flags) > > > * @v6: Set for IPv6 connections > > > * @already_sent: Number of bytes already sent > > > * @fillsize: Maximum bytes to fill in guest-side receiving window > > > - * @iov_cnt: number of iov (output) > > > - * @head_cnt: Pointer to store the count of head iov entries (outpu= t) > > > + * @elem_used: number of element (output) > > > + * @frame_cnt: Pointer to store the number of frames (output) > > > * > > > * Return: number of bytes received from the socket, or a negative = error code > > > * on failure. > > > @@ -183,57 +198,77 @@ int tcp_vu_send_flag(const struct ctx *c, struc= t tcp_tap_conn *conn, int flags) > > > static ssize_t tcp_vu_sock_recv(const struct ctx *c, struct vu_virt= q *vq, > > > const struct tcp_tap_conn *conn, bool v6, > > > uint32_t already_sent, size_t fillsize, > > > - int *iov_cnt, int *head_cnt) > > > + int *elem_used, int *frame_cnt) > > > { > > > + static struct iovec iov_msg[VIRTQUEUE_MAX_SIZE + DISCARD_IOV_NUM]; > > > const struct vu_dev *vdev =3D c->vdev; > > > struct msghdr mh_sock =3D { 0 }; > > > uint16_t mss =3D MSS_GET(conn); > > > size_t hdrlen, iov_used; > > > int s =3D conn->sock; > > > + ssize_t ret, dlen; > > > int elem_cnt; > > > - ssize_t ret; > > > - int i; > > > - > > > - *iov_cnt =3D 0; > > > + int i, j; > > > hdrlen =3D tcp_vu_hdrlen(v6); > > > + *elem_used =3D 0; > > > + > > > iov_used =3D 0; > > > elem_cnt =3D 0; > > > - *head_cnt =3D 0; > > > + *frame_cnt =3D 0; > > > while (fillsize > 0 && elem_cnt < ARRAY_SIZE(elem) && > > > - iov_used < VIRTQUEUE_MAX_SIZE) { > > > - size_t frame_size, dlen, in_total; > > > - struct iovec *iov; > > > + iov_used < ARRAY_SIZE(iov_vu) && > > > + *frame_cnt < ARRAY_SIZE(frame)) { > > > + size_t frame_size, in_total; > > > int cnt; > > > cnt =3D vu_collect(vdev, vq, &elem[elem_cnt], > > > ARRAY_SIZE(elem) - elem_cnt, > > > - &iov_vu[DISCARD_IOV_NUM + iov_used], > > > - VIRTQUEUE_MAX_SIZE - iov_used, &in_total, > > > + &iov_vu[iov_used], > > > + ARRAY_SIZE(iov_vu) - iov_used, &in_total, > > > MIN(mss, fillsize) + hdrlen, > > > &frame_size); > > > if (cnt =3D=3D 0) > > > break; > > > - assert((size_t)cnt =3D=3D in_total); /* one iovec per element */ > > > + > > > + frame[*frame_cnt].idx_element =3D elem_cnt; > > > + frame[*frame_cnt].num_element =3D cnt; > > > + frame[*frame_cnt].idx_iovec =3D iov_used; > > > + frame[*frame_cnt].num_iovec =3D in_total; > > > + frame[*frame_cnt].size =3D frame_size; > > > + (*frame_cnt)++; > > > iov_used +=3D in_total; > > > - dlen =3D frame_size - hdrlen; > > > + elem_cnt +=3D cnt; > > > - /* reserve space for headers in iov */ > > > - iov =3D &elem[elem_cnt].in_sg[0]; > > > - assert(iov->iov_len >=3D hdrlen); > > > - iov->iov_base =3D (char *)iov->iov_base + hdrlen; > > > - iov->iov_len -=3D hdrlen; > > > - head[(*head_cnt)++] =3D elem_cnt; > > > + fillsize -=3D frame_size - hdrlen; > > > + } > > > - fillsize -=3D dlen; > > > - elem_cnt +=3D cnt; > > > + /* build an iov array without headers */ > > > + for (i =3D 0, j =3D DISCARD_IOV_NUM; i < *frame_cnt && > > > + j < ARRAY_SIZE(iov_msg); i++) { > > > + struct iov_tail data; > > > + ssize_t cnt; > > > + > > > + data =3D IOV_TAIL(&iov_vu[frame[i].idx_iovec], > > > + frame[i].num_iovec, 0); > > > + iov_drop_header(&data, hdrlen); > > > + > > > + cnt =3D iov_tail_clone(&iov_msg[j], ARRAY_SIZE(iov_msg) - j, > > > + &data); > > > + if (cnt =3D=3D -1) > > > + die("Missing entries in iov_msg"); > >=20 > > Is a fatal error really what we want here? >=20 > As we are copying iov_vu into iov_msg and the sizes match (if we ignore t= he > discard part), there is always enough room. An assert() would be cleaner = but > coverity doesn't manage it correctly. Huh, weird. Okay then. --=20 David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson --C2imQKwC+qGxr/Ij Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEO+dNsU4E3yXUXRK2zQJF27ox2GcFAmoOn3QACgkQzQJF27ox 2GfqDw/+IfqtCe7b39wzk4O6x9NVnp2ZdEntVgOXJiErXZvxaShE8YS9YLeU7JW8 o5YBWBUF/+I5NVLjuwnhaLAkgdZGfv18MyupCO8d3oO5qjKhYaXk+v8gJeRFsEWC wRsa+lv4BHl+bv/U810x2OX8rKDzC0i3CRvqKL3IqgcrJ2SzrQ1vNFGJBBxptwQ+ dyAB7GTNGbTeyjQXLIxhsYKRwgcREO055WnzLgcWtVN2/EP9XyzD4cIVUxGv55+h isKoujFz8CDPAz6LXBBe50D7h5ZfbgtfmgTz5ldmqACNNBgyvkJiLzn0Bf9r4zo/ oGN99d3jZFzr2nWQp7NSEwG3wlM9pk7LNrpffqEsxnCwSyg4Nd9gPu6cywtjM3Vb s4wbeu4bK9tCs7ATJ/4prX2WXOmoDpUOF7OLcdmeqvWKteGuR1vnITctuX9ysUjV Ook7NurTLSmx4bV636BhlPlMed73eMGRSJQ8jz6jDVLV1NVPEjITQwmAnzOkSLiY Dq3sjYa5nMZceNzU9+vFSvX7NxanglR51Fk1ppu488zTkD7uTrYfLJezNVpvFZX7 97gOFe6fn2814ytNe8XPvC0yF/X/OaeZ/lypdA9PXUVHcw6nwJKNqwdI2eGWaqRl jbsOTxVimMC4Pas2r7C49uThUPeaAFNQi8u9hle8ODqxny7ctBo= =1IqT -----END PGP SIGNATURE----- --C2imQKwC+qGxr/Ij--