From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=none (p=none dis=none) header.from=gibson.dropbear.id.au Authentication-Results: passt.top; dkim=pass (2048-bit key; secure) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.a=rsa-sha256 header.s=202608 header.b=ov0Iv9fC; dkim-atps=neutral Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by passt.top (Postfix) with ESMTPS id 9F9E25A0265 for ; Mon, 03 Aug 2026 08:54:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202608; t=1785740083; bh=iKimTgxrsBWZhMLjLtjo/bvmI8g7M/ptRcVQyrmsYlA=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=ov0Iv9fCV4uoDjDDkn+R5XXo/HGWg6NNetLy+B7HAEco006acZKvP2IU/PfSKvzrz xS1MRPFNtEU1xb6YkDMr96Wk8IjIjg+0JyS7pHmKc0VyoSO4kBMy8JvVp3Rte8Ccjt MTo2XyjLgqith3rw0v5zxJ0hW4pl3mGWKZzZyJYm+CrfdcUEF/4va763a3EgJHvNEm ZLAcRi56NXe6wwvufnPzBiHLchGhDd1enBb1gEwvgCBeWQDAMwfAV5z6+MTZeckCg9 Sm4qSvwTZKvGG4K/B3YOZlGlXGEpNNygeZhK85GvmUyRA9qsxpZ15tI7WL6//KcALw Kny6rkUdaoQ5w== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4hD6qR6W9Kz4wCB; Mon, 03 Aug 2026 16:54:43 +1000 (AEST) Date: Mon, 3 Aug 2026 16:53:27 +1000 From: David Gibson To: Laurent Vivier Subject: Re: [PATCH v2 07/10] tcp: Protect init socket pools with mutex for thread safety Message-ID: References: <20260731162329.3552800-1-lvivier@redhat.com> <20260731162329.3552800-8-lvivier@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="kEuHBgkfiVuTnmmQ" Content-Disposition: inline In-Reply-To: <20260731162329.3552800-8-lvivier@redhat.com> Message-ID-Hash: 67TQ36QEFCISVVO4UOLJOYGHYRYM6OUT X-Message-ID-Hash: 67TQ36QEFCISVVO4UOLJOYGHYRYM6OUT X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --kEuHBgkfiVuTnmmQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 31, 2026 at 06:23:26PM +0200, Laurent Vivier wrote: > The pre-opened socket pools init_sock_pool4/6 are consumed by > tcp_conn_pool_sock() when creating new connections from any worker > thread, and refilled by tcp_sock_refill_pool() from tcp_timer() in > post_handler(). These can run concurrently on different threads. >=20 > Add a mutex protecting both operations in tcp_conn_sock() and > tcp_sock_refill_init(), where init namespace pools are accessed. I'm guessing you're going with a mutex rather than a per-thread socket pool for simplicity? That might be the right choice, but I do wonder a bit about it. The pool exists to avoid the latency of creating a new socket for a new connection. If the latency of taking the lock exceeds that of creating a socket, there's no longer any point to the pool. In the unconstest case, that's almost certainly not the case - a happy path futex() lock should be be much faster than a syscall. If the lock _is_ contested, I suspect opening a socket directly might win - at least for host ns sockets. For guest ns sockets, the latency is higher because we need vfork()/setns()/etc. Then again... taking a socket from the pool also involves writing the pool, which potentially means a cacheline pingpong. That might incur a pretty substantial latency. In theory we could trylock() and open a socket directly if we don't get the lock immediately, but at that point it would probably be simpler to have per-thread socket pools anyway. I guess, since there is no meaningful shared state in the socket pool, a per-thread pool seems like the more natural approach to me. >=20 > Signed-off-by: Laurent Vivier > --- > tcp.c | 10 +++++++++- > 1 file changed, 9 insertions(+), 1 deletion(-) >=20 > diff --git a/tcp.c b/tcp.c > index ab7cbfa5de83..aef689faf031 100644 > --- a/tcp.c > +++ b/tcp.c > @@ -293,6 +293,7 @@ > #include > #include > #include > +#include > =20 > #include > #include > @@ -439,6 +440,7 @@ static socklen_t tcp_info_size; > /* Pools for pre-opened sockets (in init) */ > int init_sock_pool4 [TCP_SOCK_POOL_SIZE]; > int init_sock_pool6 [TCP_SOCK_POOL_SIZE]; > +static pthread_mutex_t sock_pool_lock =3D PTHREAD_MUTEX_INITIALIZER; > =20 > /** > * conn_at_sidx() - Get TCP connection specific flow at given sidx > @@ -1581,7 +1583,11 @@ int tcp_conn_sock(sa_family_t af) > int *pool =3D af =3D=3D AF_INET6 ? init_sock_pool6 : init_sock_pool4; > int s; > =20 > - if ((s =3D tcp_conn_pool_sock(pool)) >=3D 0) > + pthread_mutex_lock(&sock_pool_lock); > + s =3D tcp_conn_pool_sock(pool); > + pthread_mutex_unlock(&sock_pool_lock); > + > + if (s >=3D 0) > return s; > =20 > /* If the pool is empty we just open a new one without refilling the > @@ -2858,6 +2864,7 @@ int tcp_sock_refill_pool(int pool[], sa_family_t af) > */ > static void tcp_sock_refill_init(const struct ctx *c) > { > + pthread_mutex_lock(&sock_pool_lock); > if (c->ifi4) { > int rc =3D tcp_sock_refill_pool(init_sock_pool4, AF_INET); > if (rc < 0) > @@ -2870,6 +2877,7 @@ static void tcp_sock_refill_init(const struct ctx *= c) > warn("TCP: Error refilling IPv6 host socket pool: %s", > strerror_(-rc)); > } > + pthread_mutex_unlock(&sock_pool_lock); > } > =20 > /** > --=20 > 2.54.0 >=20 --=20 David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson --kEuHBgkfiVuTnmmQ Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEO+dNsU4E3yXUXRK2zQJF27ox2GcFAmpwOt4ACgkQzQJF27ox 2Gf21RAAoQUvyv8VeuVKMALoedoVQzggBRPLCcovtr1ZQiUBfokTQW08UXrt8pwT ScfPDa8XjF6PRixKTG6jr8LoCwy0YWiPWOH0jPxTGnYJ/vxux1gJppDHWgjF+OAp JDtOarLGQz9xqIyG4+XM5TVH6dIhEtbWCZVJKjeuQN7j1uwvZbZb3ZR07s8nFK5Y gBgQyfcP0D9Ys0RyrxXJVOAPV/uRAzKRUTz2mLzgk6Gpl0jsDY0Fppq4G3f04itg sjGYmIVMBqX+8ceh/BhLyI1NwQBBA2YRB8UH/PhJ48HkJW26OFSpu6mJFdbxCHAW mO43kaVXp6MAft9plWbL9XyMmlstY5xx7yGbXhMzBLWfRkARi0ywuXq/kwhkgk9G wToXIOAU3F+j9zeIhEa7G+odTm03iDT3Sj/TeUuTjSq0Xt4UsHgROlBz9US6BUg5 jG31r4ALpF0jlTq5C3QqmhZ3lISiHvj+FRjLwXK9sjy2bSUyZJln/bpSI4J9vSMq swfGUxJ7/oUoxuITbr1p4T7x0OIHQzcEeSyoE3Wk9VCtQVCAbSAb3uPmTDXehDn+ iDyMK+4SefRFaTSdAMmoue5Idy8wbQqr8LeGVaOeTWVgDF4MLvmx5vCBnNf54BRF dX5fOuwz46Qt/IEW0wtDD7kaoArS3UHWzdSnfZyeb/6353OPWqI= =1Tf/ -----END PGP SIGNATURE----- --kEuHBgkfiVuTnmmQ--