From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=none (p=none dis=none) header.from=gibson.dropbear.id.au Authentication-Results: passt.top; dkim=pass (2048-bit key; secure) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.a=rsa-sha256 header.s=202608 header.b=h/6CeMOu; dkim-atps=neutral Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by passt.top (Postfix) with ESMTPS id BBA745A0269 for ; Mon, 10 Aug 2026 11:19:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202608; t=1786353538; bh=KnhubM38RP0goIQY0j4Gnj8IM7cLBzqswZzOeiitr4M=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=h/6CeMOuPws+EjNlyuNoYkwuThQ1gqiF19iHzccDFXpf83qnj195yMT5haMABRJdh SEMCbindhVr3bMKodfD+KIKHCPRRTnWQsRkJ2ZhVrjZ8lVz0sCw7L1zeCEvxHIxle3 oBjl7la8ut7lBDkDxa/5wvVhzj4exyzD02aK+6UnHo5Da8/eMxFb/c5/dnsq356OBw EJts98nvSk+3muP5sODhoCxS2B0pxhgrSrjIp7JN1YYdFRUxND3+JMnPIDkFuxNcgm ZWZ1N0UIFeg7YFWCVc/bsjFs1xXWhB62ajfP6NOKUyuJojO/h6OXb+5Z2M8cbymEj5 C5mBo8bFPEwvg== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4hJThf0ZBbz4w9r; Mon, 10 Aug 2026 19:18:58 +1000 (AEST) Date: Mon, 10 Aug 2026 19:18:53 +1000 From: David Gibson To: Ammar Yasser Subject: Re: [RFC v3 8/8] tap: Implement pasta vhost TX Message-ID: References: <20260802132155.870796-1-aerosound161@gmail.com> <20260802132155.870796-9-aerosound161@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="CkNUBXEkxA34TDp7" Content-Disposition: inline In-Reply-To: <20260802132155.870796-9-aerosound161@gmail.com> Message-ID-Hash: DBGCVGK2O2KZ5P7PSENHQQJWND7SSIPF X-Message-ID-Hash: DBGCVGK2O2KZ5P7PSENHQQJWND7SSIPF X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top, eperezma@redhat.com X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --CkNUBXEkxA34TDp7 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Aug 02, 2026 at 01:21:55PM +0000, Ammar Yasser wrote: > Callers will specify whether vhost should be used or no by passing the > vhost argument to tap_send_frames_pasta. If specified, sending will go > through a function called tap_send_frames_vhost, which pops a descriptor > from the queue shared with the kernel and sets the address of that > descriptor to be the base address of a single iov from the group of > buffers_per_frame * nframes iovs we pass to the function >=20 > Signed-off-by: Ammar Yasser Sorry, didn't spot this earlier in the series: if these are based on Eugenio's earlier patches, they should have his Signed-off-by in addition to your own. > --- > tap.c | 108 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++- > 1 file changed, 107 insertions(+), 1 deletion(-) >=20 > diff --git a/tap.c b/tap.c > index e177eef..66bb71d 100644 > --- a/tap.c > +++ b/tap.c > @@ -361,7 +361,32 @@ void tap_icmp6_send(const struct ctx *c, > } > =20 > /** > - * tap_send_frames_pasta() - Send multiple frames to the pasta tap > + * tx_reap() - Reclaim the descriptors the kernel has already processed > + */ > +static void tx_reap(void) { > + struct vring_used *used =3D &vring_used_all[1].used; > + uint16_t used_idx =3D le16toh(used->idx); > + > + smp_rmb(); > + > + /* increment last_used_idx until it reaches the kernel's used index */ > + while (vqs[1].last_used_idx !=3D used_idx) { > + uint16_t desc_id =3D le32toh(used->ring[vqs[1].last_used_idx % VHOST_N= DESCS].id); > + =09 > + for (;;) { > + /* keep going until we find a descriptor without the next flag */ > + vqs[1].num_free++; > + if (!(le16toh(vring_desc[1][desc_id].flags) & VRING_DESC_F_NEXT)) > + break; > + /* this descriptor wasn't the last, set desc_id to the next one and k= eep going */ > + desc_id =3D le16toh(vring_desc[1][desc_id].next); > + } > + vqs[1].last_used_idx++; > + } > +} > + > +/** > + * tap_send_frames_vhost() - Send multiple frames to the pasta tap > * @c: Execution context > * @iov: Array of buffers > * @bufs_per_frame: Number of buffers (iovec entries) per frame > @@ -371,6 +396,84 @@ void tap_icmp6_send(const struct ctx *c, > * @bufs_per_frame contiguous buffers representing a single frame. > * > * Return: number of frames successfully sent > + */ > +static size_t tap_send_frames_vhost(const struct ctx *c, > + const struct iovec *iov, > + size_t bufs_per_frame, size_t nframes) > +{ > + size_t i; > + size_t processed_frames =3D 0; Reverse christmas tree. > + > + /* update our local counters first */ > + tx_reap(); > + > + #define AVAIL_Q(i)(vring_avail_all[i].avail) # for preprocessor directives should always go in column 0. > + > + for (i =3D 0; i < nframes; i++) { > + size_t j; > + > + if (vqs[1].num_free < bufs_per_frame) > + break; IIRC, we were discussing during the last call that at least for the first version it's fine if we synchronously wait for buffers to be available. Since then it occurred to me that another acceptable option would be to simply drop frames if there are no available buffers (this is kind of how IP is designed to work - congestion is reported implicitly as packet loss). Do whichever is easier for now and it can be refined later. > + > + /* set the index of the avail ring in the tx queue to be our last_used= _idx */ > + uint16_t head =3D vqs[1].next_free % VHOST_NDESCS; No inline decls. > + AVAIL_Q(1).ring[(AVAIL_Q(1).idx + i) % VHOST_NDESCS] =3D htole16(head); > + > + /* we will be consuming bufs_per_frame descriptors for every frame, de= crement the local num_free */ > + vqs[1].num_free -=3D bufs_per_frame; > + > + for (j =3D 0; j < bufs_per_frame; ++j) { > + /* get the last_used_idx descriptor */ > + struct vring_desc *desc =3D &vring_desc[1][vqs[1].next_free % VHOST_N= DESCS]; > + /* > + * the iov variable contains the iovecs for all frames we will send. > + * access a single fragment of a frame (each fragment is one of tcp_i= ov_parts) > + * denoted by iov at index i (index of the frame being processed) * b= ufs_per_frame=20 > + * plus j (the index of the fragment being processed) > + */=09 > + const struct iovec *iov_i =3D &iov[i * bufs_per_frame + j]; > + > + /*=20 > + * set that descriptor's address to the base of the iov and set the V= RING_DESC_F_NEXT > + * flag on the descriptor if its not the last frame fragment, so that= the > + * guest would recieve the entire frame in one go.=20 > + */ > + desc->addr =3D (uint64_t)iov_i->iov_base; > + desc->len =3D iov_i->iov_len; > + desc->flags =3D (j =3D=3D bufs_per_frame - 1) ? 0 : htole16(VRING_DES= C_F_NEXT); > + vqs[1].next_free++; > + } > + > + processed_frames++; > + } > + > + /* we didn't process any frames, no need to notify the kernel */ > + if ((processed_frames =3D=3D 0)) > + return 0; > + > + smp_wmb(); > + /* we will have used nframes descriptor chains */ > + AVAIL_Q(1).idx =3D htole16(le16toh(AVAIL_Q(1).idx)+processed_frames); > + #undef AVAIL_Q > + > + vhost_kick(&vring_used_all[1].used, c->vq[1].kick_fd); > + > + return processed_frames; > +} > + > + > +/** > + * tap_send_frames_pasta() - Send multiple frames to the pasta tap > + * @c: Execution context > + * @iov: Array of buffers > + * @bufs_per_frame: Number of buffers (iovec entries) per frame > + * @nframes: Number of frames to send > + * @vhost: Use vhost-kernel or not > + * > + * @iov must have total length @bufs_per_frame * @nframes, with each set= of > + * @bufs_per_frame contiguous buffers representing a single frame. > + * > + * Return: number of frames successfully sent (or queued) > * > * #syscalls:pasta write > */ > @@ -381,6 +484,9 @@ static size_t tap_send_frames_pasta(const struct ctx = *c, > size_t nbufs =3D bufs_per_frame * nframes; > size_t i; > =20 > + if (vhost) > + return tap_send_frames_vhost(c, iov, bufs_per_frame, nframes); > + > for (i =3D 0; i < nbufs; i +=3D bufs_per_frame) { > ssize_t rc =3D writev(c->fd_tap, iov + i, bufs_per_frame); > size_t framelen =3D iov_size(iov + i, bufs_per_frame); > --=20 > 2.34.1 >=20 --=20 David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson --CkNUBXEkxA34TDp7 Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEO+dNsU4E3yXUXRK2zQJF27ox2GcFAmp5l3wACgkQzQJF27ox 2GewSQ//SBVUtl71ofaFg/HskA7zMYvvxVxJN0zTDl+6/UA9hZomfzjQuNviKvrj 0xu+xWYMh0oM/Vp7TwCKSBX1bZVF9UeRXhh1nSKgVXXIA9GGFAnsXYvxa2T8uHX6 sA7fDa2YGshLxp8KWhWsnrLknynzS6+qYuhsEgYate2RSvqIvqDYxx6n5v99g8zY KxDiaSTEKvkiLv9iCfYvZpC/v1kC5PC/gdO+Yi9/qEPMI9JXI8gp36vq8Z4EjfI6 X/sfNwDc6320T2LXTvyz8n31N7FmsvyrZNIv8s7SRd7XFNQ9Qd47sC5YvunpMPKO HXS1zUWP1IgWIDM6mTxF5rBWrcW0uLxiRl90n1sKw8AvvzaJ6tdk0iAbYdi8Ojnt IuIaok387+/BMi5PDVowkLp6+a+YVA2dszd6ZW68DWuYfTv+V9+AOh5zuTifY7VL DSA2m3gaFyIhZ5l2QbllLy8nIaKwFEH5vM5BHnZcGPK1oTuRsPHhzEyTyAbs/JYQ b+J6FL9lmQobm1AV8nwNRCrXYIOVq7A03d+jmBgQL1r4YSiWWXxQreQvu5JPL4X2 qVXqS4GDdNAK+VUfLncGBMlSG1u4vAdqXlvyz+WMPszJP8vkmX14z6gqS6NXV/kJ 8DRxFO3trRGVzbtU5nmGebU1q6PN3SI6UZ9p6G0PqnJoy4Be1ck= =ZthX -----END PGP SIGNATURE----- --CkNUBXEkxA34TDp7--