Re: [PATCH 3/5] tcp, flow: Replace per-connection in_epoll flag with epollfd in flow_common

[Laurent Vivier <lvivier@redhat.com>]

On 07/10/2025 08:07, David Gibson wrote:
> On Fri, Oct 03, 2025 at 05:27:15PM +0200, Laurent Vivier wrote:
>> The in_epoll boolean flag in tcp_tap_conn and tcp_splice_conn only tracked
>> whether a connection was registered with epoll, not which epoll instance.
>> This limited flexibility for future multi-epoll support.
>>
>> Replace the boolean with an epollfd field in flow_common that serves dual
>> purpose: zero indicates not registered (replacing in_epoll=false), non-zero
> 
> Don't use 0, since that's a valid fd.
> 
>> stores the actual epoll fd (replacing in_epoll=true).
> 
> I am a bit nervous about adding 31-bits to every flow, since I think
> we're fairly close to a cacheline threshold.
> 
> I'm not sure we really can add any less to flow_common, though, given
> alignment.
> 
> Then again... we probably don't need 8 bites each for TYPE and STATE,
> so those could be packed tighter.  Then we could use a limited-bits
> index into a table of epollfds, rather than a raw fd.  Much uglier,
> but maybe worth it?
> 

I tried the epollfds table. But it introduces complexity and a new shared data structure 
that we will need to manage correctly in the case of multithreading. I don't think it's 
the way to follow...

The epollfds will be open at startup (we have already one: c->epollfd) and their number 
will depend on the number of threads/queues (the maximum number of virtqueues for QEMU is 
65536 but the number of queues is generally configured to match the number of vCPU). And 
we can guess the file descriptor will be allocated in order. So I think we can expect to 
fit in an 8-bit. In passt I'm setting the maximum number of virtqueues to 64, that will 
limit also the number of epollfd.

Thanks,
Laurent