Re: [PATCH 3/5] tcp, flow: Replace per-connection in_epoll flag with epollfd in flow_common

[David Gibson <david@gibson.dropbear.id.au>]

[-- Attachment #1: Type: text/plain, Size: 2341 bytes --]

On Tue, Oct 07, 2025 at 03:26:23PM +0200, Laurent Vivier wrote:
> On 07/10/2025 08:07, David Gibson wrote:
> > On Fri, Oct 03, 2025 at 05:27:15PM +0200, Laurent Vivier wrote:
> > > The in_epoll boolean flag in tcp_tap_conn and tcp_splice_conn only tracked
> > > whether a connection was registered with epoll, not which epoll instance.
> > > This limited flexibility for future multi-epoll support.
> > > 
> > > Replace the boolean with an epollfd field in flow_common that serves dual
> > > purpose: zero indicates not registered (replacing in_epoll=false), non-zero
> > 
> > Don't use 0, since that's a valid fd.
> > 
> > > stores the actual epoll fd (replacing in_epoll=true).
> > 
> > I am a bit nervous about adding 31-bits to every flow, since I think
> > we're fairly close to a cacheline threshold.
> > 
> > I'm not sure we really can add any less to flow_common, though, given
> > alignment.
> > 
> > Then again... we probably don't need 8 bites each for TYPE and STATE,
> > so those could be packed tighter.  Then we could use a limited-bits
> > index into a table of epollfds, rather than a raw fd.  Much uglier,
> > but maybe worth it?
> > 
> 
> I tried the epollfds table. But it introduces complexity and a new shared
> data structure that we will need to manage correctly in the case of
> multithreading. I don't think it's the way to follow...

I was assuming it would be a write-once data structure, meaning
multithreaded management shouldn't be too hard.  But ok.

> The epollfds will be open at startup (we have already one: c->epollfd) and
> their number will depend on the number of threads/queues (the maximum number
> of virtqueues for QEMU is 65536 but the number of queues is generally
> configured to match the number of vCPU). And we can guess the file
> descriptor will be allocated in order. So I think we can expect to fit in an
> 8-bit.

Right, I guess it will.  I don't love relying on that, but it's
certainly an option.

> In passt I'm setting the maximum number of virtqueues to 64, that
> will limit also the number of epollfd.
> 
> Thanks,
> Laurent
> 
> 

-- 
David Gibson (he or they)	| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you, not the other way
				| around.
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]