From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=none (p=none dis=none) header.from=gibson.dropbear.id.au Authentication-Results: passt.top; dkim=pass (2048-bit key; secure) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.a=rsa-sha256 header.s=202606 header.b=aXlgbW5j; dkim-atps=neutral Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by passt.top (Postfix) with ESMTPS id B14C65A0269 for ; Thu, 02 Jul 2026 05:42:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202606; t=1782963771; bh=z0mi/WKMFyqDFE7mLnl/hO6WiIVMjJRXWk3IxDbLOv0=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=aXlgbW5jTv1MXRMBFfDdOxUcgZxiLXu3p7259J34RNH1X4ann5jENOk+ZjasWPuUw KXqXM+l9EZQpE0TxFawEKTeL7s1F9bofJkl7SFnaALhe8ebpGlv5h4Yzd+hnzY46BC 2fr19JqQDzly+eKOzjWM6Aq2uRUhu7jK+b8uoFaBPJpHrBNMSg6kitpW5W1pSYt2Kp MBbrB9F6oFKK+UKjLzzmx1MUzkW2PkociZKVJI1G/XTj+jCaDxXYj0qM/GKN31iEP3 242jvEgfcaij/wtRDa7GNYK9AfmNyyYW+l1hQkHgrNI+yOAK3+sNJK557n4IiyoqRx +hM3Y7nOx2Ftw== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4grN4q4zSKz4x01; Thu, 02 Jul 2026 13:42:51 +1000 (AEST) Date: Thu, 2 Jul 2026 12:59:47 +1000 From: David Gibson To: Laurent Vivier Subject: Re: [PATCH 7/8] tcp: Protect init socket pools with mutex for thread safety Message-ID: References: <20260616171052.3785909-1-lvivier@redhat.com> <20260616171052.3785909-8-lvivier@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="p54QCRYOResojXPb" Content-Disposition: inline In-Reply-To: <20260616171052.3785909-8-lvivier@redhat.com> Message-ID-Hash: 5TYG3Q65J33AZC5KPUUKG5MGFOZ3EPJG X-Message-ID-Hash: 5TYG3Q65J33AZC5KPUUKG5MGFOZ3EPJG X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --p54QCRYOResojXPb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jun 16, 2026 at 07:10:51PM +0200, Laurent Vivier wrote: > The pre-opened socket pools init_sock_pool4/6 are consumed by > tcp_conn_pool_sock() when creating new connections from any worker > thread, and refilled by tcp_sock_refill_pool() from tcp_timer() in > post_handler(). These can run concurrently on different threads. >=20 > Add a mutex protecting both operations in tcp_conn_sock() and > tcp_sock_refill_init(), where init namespace pools are accessed. I wonder if we might be better off with a per-qpair/thread socket pool instead of a mutexed common pool. The obvious tradeoff would be that the pool is smaller so a thread could run out of sockets while we still have some available on another thread. But, having a common pool only gains us there if the mutex lock is significantly cheaper than the system calls to refill the pool. That might be the case, but I'm not certain. >=20 > Signed-off-by: Laurent Vivier > --- > tcp.c | 10 +++++++++- > 1 file changed, 9 insertions(+), 1 deletion(-) >=20 > diff --git a/tcp.c b/tcp.c > index 955012355d69..019340c1c348 100644 > --- a/tcp.c > +++ b/tcp.c > @@ -293,6 +293,7 @@ > #include > #include > #include > +#include > =20 > #include > #include > @@ -439,6 +440,7 @@ static socklen_t tcp_info_size; > /* Pools for pre-opened sockets (in init) */ > int init_sock_pool4 [TCP_SOCK_POOL_SIZE]; > int init_sock_pool6 [TCP_SOCK_POOL_SIZE]; > +static pthread_mutex_t sock_pool_lock =3D PTHREAD_MUTEX_INITIALIZER; > =20 > /** > * conn_at_sidx() - Get TCP connection specific flow at given sidx > @@ -1568,7 +1570,11 @@ int tcp_conn_sock(sa_family_t af) > int *pool =3D af =3D=3D AF_INET6 ? init_sock_pool6 : init_sock_pool4; > int s; > =20 > - if ((s =3D tcp_conn_pool_sock(pool)) >=3D 0) > + pthread_mutex_lock(&sock_pool_lock); > + s =3D tcp_conn_pool_sock(pool); > + pthread_mutex_unlock(&sock_pool_lock); > + > + if (s >=3D 0) > return s; > =20 > /* If the pool is empty we just open a new one without refilling the > @@ -2857,6 +2863,7 @@ int tcp_sock_refill_pool(int pool[], sa_family_t af) > */ > static void tcp_sock_refill_init(const struct ctx *c) > { > + pthread_mutex_lock(&sock_pool_lock); > if (c->ifi4) { > int rc =3D tcp_sock_refill_pool(init_sock_pool4, AF_INET); > if (rc < 0) > @@ -2869,6 +2876,7 @@ static void tcp_sock_refill_init(const struct ctx *= c) > warn("TCP: Error refilling IPv6 host socket pool: %s", > strerror_(-rc)); > } > + pthread_mutex_unlock(&sock_pool_lock); > } > =20 > /** > --=20 > 2.54.0 >=20 --=20 David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson --p54QCRYOResojXPb Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEO+dNsU4E3yXUXRK2zQJF27ox2GcFAmpF1BcACgkQzQJF27ox 2GeW6g//Q1zTgyNyPL07WGGIGBTF16XO9HHJC9g3GptNxB1ic2dCPZK4XFTTWKkK /5Ei3429x6bngTgJ9Jdvus8fEdUOhUK7Ws9xOhSS2eBvxpCUeeGiadImDvPP0R1X BryF7Ak9lFpGUqz88ClKskaYX0IDth7I2y6gFcHXXWr6GGhDdzhZta1hUqNc/jZe Qg2bH86bm1y50MooNEHau/DfRBo39WWRDr0vWIte3ULVuQImXGTPVqVXgImqO6x+ qRK98sPkajyeuRE8aphMy8VdzOZr6PO7LkDGEsLkAeZzCgUN55bnojAR/Uh+WH7C SzV6SQtid5w0yyjZGAqNWeUIdg8Pc7jxi++m5UvXUTkhimkIGgk1GTlYKOIC0rK1 G0vWfKZMKpSw7SNpNd3KybTSFrExbU3vRorRMtY0PlSc4HW4fDq+WVgKq6PVvLhA sMHvMjMB8Owxz7i98aRAr0PKlNuITKEycKgVVL+FfktaXEn6t7spcoYlB6Hln9ag FbmZ8knLZKCSt+29XZGZfkV4xQ5PW8irlbVNe97Vmllj9/ArS82KyzXMnIop3EhI 3/nYKKLaTeZ7mpZCDonai9+zr4bnWt/IBj9IiMUcV7oPPMLA7anLPpBei9xQQoTc o4r2DecfbHKNICB6a2J1CaXjCLYdo4HAgKpqXMvP836b3D4k80k= =AwUk -----END PGP SIGNATURE----- --p54QCRYOResojXPb--