From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=none (p=none dis=none) header.from=gibson.dropbear.id.au Authentication-Results: passt.top; dkim=pass (2048-bit key; secure) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.a=rsa-sha256 header.s=202506 header.b=NrSAH9jr; dkim-atps=neutral Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by passt.top (Postfix) with ESMTPS id 0EE055A027B for ; Tue, 29 Jul 2025 02:32:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202506; t=1753748965; bh=4sPikmujmZBCFDD9PAu0Bm0Xw3/F7Gwz9M7h6RnSI2s=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=NrSAH9jrie0f1N9NHFXa+KNXnQ/I49bCBgs6tiUYXmkdNosna5moZYxKvASnHuWfh YVYVEN7PoYy+JLoMn30ALWAoTXDWltgj5E7GCZlJRGo86Jy9im7QMI4ZCybcgW3osV T5ykCW28W4oFLtUblx40mFZHA+RBENrCodiDzGH0d7qc2aD9KqgDQbRTXcBrbutNh0 OL1H/NwoAp+cFZigpJU2shERMOz/tNGHl8M1X+IAnywB9iH0YwVmISBSToHFUDcYxL 16z5ri7Z/1dgdH2l0BGBlpYhilhR7+o1dai4tikXZ5m/rOCzBZDBVJK0AzyLZMVAt7 1+dyWZNAuKojg== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4brbnd3JCVz4w2S; Tue, 29 Jul 2025 10:29:25 +1000 (AEST) Date: Tue, 29 Jul 2025 10:30:43 +1000 From: David Gibson To: Eugenio Perez Martin Subject: Re: [RFC v2 09/11] tcp: start conversion to circular buffer Message-ID: References: <20250709174748.3514693-1-eperezma@redhat.com> <20250709174748.3514693-10-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="aNahuozEDx8ervOh" Content-Disposition: inline In-Reply-To: Message-ID-Hash: QK6JKYVWMSIO6WPYEH2GXANVIAGBA5RI X-Message-ID-Hash: QK6JKYVWMSIO6WPYEH2GXANVIAGBA5RI X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top, jasowang@redhat.com X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --aNahuozEDx8ervOh Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 28, 2025 at 06:55:50PM +0200, Eugenio Perez Martin wrote: > On Thu, Jul 24, 2025 at 3:03=E2=80=AFAM David Gibson > wrote: > > > > On Wed, Jul 09, 2025 at 07:47:46PM +0200, Eugenio P=C3=A9rez wrote: > > > The vhost-kernel module is async by nature: the driver (pasta) places= a > > > few buffers in the virtqueue and the device (vhost-kernel) trust the > > > > s/trust/trusts/ > > >=20 > Fixing in the next version. >=20 > > > driver will not modify them until it uses them. To implement it is n= ot > > > possible with TCP at the moment, as tcp_buf trust it can reuse the > > > buffers as soon as tcp_payload_flush() finish. > > > > > > > > > > > > To achieve async let's make tcp_buf work with a circular ring, so vho= st > > > can transmit at the same time pasta is queing more data. When a buff= er > > > is received from a TCP socket, the element is placed in the ring and > > > sock_head is moved: > > > [][][][] > > > ^ ^ > > > | | > > > | sock_head > > > | > > > tail > > > tap_head > > > > > > When the data is sent to vhost through the tx queue, tap_head is moved > > > forward: > > > [][][][] > > > ^ ^ > > > | | > > > | sock_head > > > | tap_head > > > | > > > tail > > > > > > Finally, the tail move forward when vhost has used the tx buffers, so > > > tcp_payload (and all lower protocol buffers) can be reused. > > > [][][][] > > > ^ > > > | > > > sock_head > > > tap_head > > > tail > > > > This all sounds good. I wonder if it might be clearer to do this > > circular queue conversion as a separate patch series. I think it > > makes sense even without the context of vhost (it's closer to how most > > network things work). > > >=20 > Sure it can be done. >=20 > > > In the case of error queueing to the vhost virtqueue, sock_head moves > > > backwards. The only possible error is that the queue is full, as > > > > sock_head moves backwards? Or tap_head moves backwards? > > >=20 > Sock head moves backwards. Tap_head cannot move backwards as vhost > does not have a way to report "the last X packets has not been sent". Right, I realised that as I read further. > > > virtio-net does not report success on packet sending. > > > > > > Starting as simple as possible, and only implementing the count > > > variables in this patch so it keeps working as previously. The circu= lar > > > behavior will be added on top. > > > > > > From ~16BGbit/s to ~13Gbit/s compared with write(2) to the tap. > > > > I don't really understand what you're comparing here. >=20 > Sending through vhost-net vs write(2) to tap device. Ok. That's a bit dissapointing. > > > Signed-off-by: Eugenio P=C3=A9rez > > > --- > > > tcp_buf.c | 63 +++++++++++++++++++++++++++++++++++------------------= -- > > > 1 file changed, 40 insertions(+), 23 deletions(-) > > > > > > diff --git a/tcp_buf.c b/tcp_buf.c > > > index 242086d..0437120 100644 > > > --- a/tcp_buf.c > > > +++ b/tcp_buf.c > > > @@ -53,7 +53,12 @@ static_assert(MSS6 <=3D sizeof(tcp_payload[0].data= ), "MSS6 is greater than 65516") > > > > > > /* References tracking the owner connection of frames in the tap out= queue */ > > > static struct tcp_tap_conn *tcp_frame_conns[TCP_FRAMES_MEM]; > > > -static unsigned int tcp_payload_used; > > > +static unsigned int tcp_payload_sock_used, tcp_payload_tap_used; > > > > I think the "payload" here is a hangover from when we had separate > > queues for flags-only and data-containing packets. We can probably > > drop it and make a bunch of names shorter. >=20 > Maybe we can short even more if we isolate this in its own > circular_buffer.h or equivalent. UDP will also need it. Maybe, yes. --=20 David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson --aNahuozEDx8ervOh Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEO+dNsU4E3yXUXRK2zQJF27ox2GcFAmiIFjMACgkQzQJF27ox 2GdAQw//fICNTmqLQ3IU4a3PQ7qqJwFDEqeI0+1eHl4wQv/jo2zvmFKKYEyV0hPB Dp9Fu6Aw7E9mFUZt9Syx9okfZAPCF94EDF2u6tkxTovnl+xhmr7WW1jdw+KS5fHX a3uelcVi7aMjwlZv/mVvkuT9L4isPMcpE7/okp4DbdDbDjHAvc9KwRwD3CGDd9wl 6R9Zi8+L818riI5V/1/rxANH1hAPSbPCtWraZ3q+8Gobld77UJazzfMFDkI3y9fU oST3cQlYBInx17gf0HZeOjvk/yHtCW6KabvhUvzHnKeAPXvDC79KieDnIit8C2Iw ejPMWWxboPfdE7I/v3sF2MHCHi7bm4h+gH/GRznxhBcUNQymsLIoorMSwuffNbE1 GEnAoDz7njIGCqYVgvGul77ca32pfgeViRXtytDBzftNUkskmcZjK0K5/nn6m0vc vUbHlnqFCMHmBbYl4dCFL1lgoMPkSJ8fxi+xdzqsa7kzxgjGJzvB/jyVJxHwNfA8 vIeZkAMQe3lmebAWA8G+V6D86hC4N4Ar46sZU3802GpzePFD8pj6EVsKJStlRvRX zMnDTR7dgZC0JI7sfsUeZMITnCRSS5tYlq0kNEdRUR3BcqUUGZ0lKONwfWa6FhqL GHGceb4OQtG+TBkH/MYrCcAzSDXT3RdtbB/rcAAp8hjei053z6w= =s4vi -----END PGP SIGNATURE----- --aNahuozEDx8ervOh--