From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=none (p=none dis=none) header.from=gibson.dropbear.id.au Authentication-Results: passt.top; dkim=pass (2048-bit key; secure) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.a=rsa-sha256 header.s=202606 header.b=BwwijvK2; dkim-atps=neutral Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by passt.top (Postfix) with ESMTPS id CECA75A026D for ; Mon, 22 Jun 2026 04:40:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202606; t=1782096000; bh=kl7aj3NZ6CR28AI9q57Pd83oLJ0vuaMSjzDydvHzPYY=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=BwwijvK2V0VewGWjb6Fg8paVQw0mPt2PjLR7cK6Y2nvhbQwMGcEtjDmhIXN6yRzqs mqWn/JdBlrWYiJcJZapbMshULdtUsAvSTKEQk74qIMdPwzLb0u2l2iUvEKXU1bsMHD mLqFrj6wmrVciz+aaYPn5auz96ZY2QB1B+TkG1LxxuHJnEqrY/bFnsVKmOJ24GwXX2 VvfZZb0QT/UdeeoETV2LqUBdXsLq65lSeTB2w18jN2BktDxlcoIlVQb5IBCVz+BvoU NNnlRLUx2GrjMR4y1+5z72gJeGqOxu5TZI8NXiUGlRZggS1Jrbfx+VB+ik0qxcyhp6 kB6/qUTo1B89g== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4gkC8w4GmNz4wMG; Mon, 22 Jun 2026 12:40:00 +1000 (AEST) Date: Mon, 22 Jun 2026 12:36:03 +1000 From: David Gibson To: Laurent Vivier Subject: Re: [PATCH v5 06/12] tcp: Pass queue pair explicitly through TCP send path Message-ID: References: <20260616125130.1324274-1-lvivier@redhat.com> <20260616125130.1324274-7-lvivier@redhat.com> <499724cc-6262-402d-9e63-b38ec171a7b2@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="0H9/oYzAN9to+MbS" Content-Disposition: inline In-Reply-To: <499724cc-6262-402d-9e63-b38ec171a7b2@redhat.com> Message-ID-Hash: XZTZ7VARG3ZFZK7EJYK7Q2RFXMSXNH6H X-Message-ID-Hash: XZTZ7VARG3ZFZK7EJYK7Q2RFXMSXNH6H X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --0H9/oYzAN9to+MbS Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jun 19, 2026 at 07:07:56PM +0200, Laurent Vivier wrote: > On 6/19/26 08:00, David Gibson wrote: > > On Tue, Jun 16, 2026 at 02:51:24PM +0200, Laurent Vivier wrote: > > > Thread a qpair parameter from the entry points (tcp_sock_handler, > > > tcp_timer_handler, tcp_tap_handler, tcp_defer_handler) through every > > > intermediate function down to the vhost-user send functions, so calle= rs > > > explicitly select the target RX virtqueue instead of hardcoding > > > QPAIR_DEFAULT. > > >=20 > > > Add a qpair parameter to tcp_send_flag(), tcp_data_from_sock(), > > > tcp_rst_do() and its tcp_rst() macro, tcp_rewind_seq(), > > > tcp_data_from_tap(), tcp_conn_from_sock_finish(), tcp_connect_finish(= ), > > > tcp_tap_window_update(), tcp_conn_from_tap(), tcp_rst_no_conn(), > > > tcp_keepalive(), and tcp_inactivity(). > >=20 > > For the to-guest functions which take a connection parameter, this > > seems odd to me. Can't they deduce the right queue from the > > connection? >=20 > The connection's qpair (conn->f.qpair) can change at any time when > another thread processes a tap packet for the same flow and calls > FLOW_MIGRATE(). Ahh. Hmm. Right. I guess I don't really know the concurrency model you're going for. I had assumed that each flow was pinned to a thread, and migrating it was an operation requiring a maybe complex and heavyweight synchronization step. > If a socket event fires on qpair 0's epoll instance and we read > conn->f.qpair during processing, another thread might change it > concurrently via FLOW_MIGRATE(). The qpair from the epoll event is > the stable, race-free reference for "which queue am I operating on." Ok. For call chains initiated on the tap side, that's clear enough - it's the queue we got the initiating event on. For things initiaed on the socket side it's less clear what "queue I am operating on" means. I guess it means the queue associated with the epoll set tne initiating event occurred on? > So the explicit parameter is intentional: it carries the qpair from > the epoll event through the call chain, independent of the mutable > flow state. I guess I haven't looked at the multithread series yet, but it feels like the picture has a gap. In general we freely access fields in the connection. I had assumed that meant it was "owned" by the current thread. If the owning thread can change midstream, don't we need some other sort of synchronization accessing *anything* in the flow entry? --=20 David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson --0H9/oYzAN9to+MbS Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEO+dNsU4E3yXUXRK2zQJF27ox2GcFAmo4n4UACgkQzQJF27ox 2GezKA/+LJY1uhSOEELrG3xi3p73V+11gbldjqMMBEQyNvianwEs/Pt8jgdzNXAi B6RTKD/mJ2DKSEqBzWHJVDY1gN+GVm68Jy88aMH5MtybnmSRyQegG2/JfHsOcB6o Z78wPd2SIBteg0bamh/gO7gQGMPp4kDwvujdQdExLBKYPNVIekMVceS1bJZ2mLmH 8wq90Dn77++4TzC2Z0L5pwQy8z48epCEPT4pCHtUYop9u7XPvqkI2GdKBBPJom2e nAKvJVbTRFOooOehl3eYLf13mXq/tNGylJLD9i0/axRJReDiF4y8dYzCP86FSwMB X+E6r79Vcm1Ss80+msWmyJCIZ9naFo4oUBf5Lx8BMR56597cPQV5Vb8PWvJ3z+IY bkpyoDVI2K+b11pvOuU5OQ63A2XRPMCvgWoqXNZAsi6oAHnzgOt7MjnK8cCepIpa aW7zhuSzn2k+HosBocQsboGsRZNXjSUUdqnbY1zERXBoXzOJ1TnvKrwXzVU8sRJy niQBlqlxSIvk/xamTAg3iPiRrqCrNg6qIE3a+Kind8uzDLO30G2cuXgFOpnJ9J8N Jw7DEn8QhbqUu7fpFGjoHBQhNB8FHYejY+DbGpRFBsqX+x+M+lssPJl2BixMtrBL vOo2rQyb2jislkPqBMPpFAIdjB/pc9WETAoKsDCzqf9mOdHfnI8= =WDsp -----END PGP SIGNATURE----- --0H9/oYzAN9to+MbS--