Re: [PATCH 5/5] fuzz: Add test server for bidirectional protocol fuzzing

[David Gibson <david@gibson.dropbear.id.au>]

[-- Attachment #1: Type: text/plain, Size: 13620 bytes --]

On Fri, Aug 14, 2026 at 09:35:59AM +0200, Stefano Brivio wrote:
> On Fri, 14 Aug 2026 15:40:35 +1000
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > On Thu, Aug 13, 2026 at 09:53:24AM +0200, Stefano Brivio wrote:
> > > On Wed, 12 Aug 2026 12:56:28 +0530
> > > Anshu Kumari <anskuma@redhat.com> wrote:
> > >   
> > > > Add fuzz-server that acts as passt's network peer
> > > > during fuzzing.  
> > > 
> > > To me, this part makes sense. But this one:
> > >   
> > > > It connects to passt's UNIX socket  
> > > 
> > > much less, while:
> > >   
> > > > and listens on 127.0.0.1:9999 for TCP connections.  
> > > 
> > > this is the part that I expected instead. Otherwise it's not just
> > > passt's network peer, it's the guest as well.
> > >   
> > > > UNIX socket path: responds to ARP requests and TCP SYNs with
> > > >   stateless replies (swapped addresses, fixed ISN). Responses
> > > >   are XOR'd with AFL++ shared memory data so the fuzzer can
> > > >   mutate server behavior.  
> > > 
> > > This looks rather complicated to me.  
> > 
> > It does.
> > 
> > > The approach I was suggesting with a test server is the following:
> > > 
> > > 
> > >     ,- exchanges guest-side data with ------------.
> > >     |                                   ,---------|---------.
> > >     |                                ,--|      passt        |
> > >     |                               /   '-.---------------^-'
> > > ,---|---.                          /      | connect(),    | accept(),
> > > | AFL++ |-- shares memory with ---|       | send data,    | reply with
> > > '---|---'                          \      | etc.          | data, etc.
> > >     |                               \   ,-v---------------'-.
> > >     |                                '--|    test server    |
> > >     |                                   '---------|---------'
> > >     '- exchanges host-side data with -------------'
> > > 
> > > ...at least in its basic form. Eventually, the test server should be
> > > able to connect to passt itself (and we could call it "test peer" at
> > > that point).  
> > 
> > So, I agree that to meaningfully fuzz things, we want the fuzzer to be
> > able to control data on both the guest and host side.  I can see two
> > basic approaches:
> > 
> >  A) Alter passt/pasta so that instead of directly communicating with
> >     external entities (either guest or host side) we use mocked
> >     versions which retrieve data from AFL.  We can do that either at
> >     the system call level, or at a higher helper function level, the
> >     lower level we go, the more of the "normal" passt code we're
> >     exercising, but doing at a slightly higher level might be easier
> >     to implement
> > 
> >  B) Run passt/pasta in an environment where we can intercept the
> >     external transfers to a test server / test peer / test guest which
> >     in turn responds based on data from AFL.
> > 
> > Both the current draft and the sketch diagram Stefano has provided are
> > a hybrid of both approaches, so far I'm not seeing a clear advantage
> > to that over going all one way or the other.
> 
> Of course, B) would be cleaner (and not that complicated, see below),
> but we don't want to do that guest-side (sending data from a test
> guest) because we would lose the speed advantage of having shared
> memory on the path that _really_ matters for fuzzing (the guest is
> untrusted, the kernel isn't).

I agree we lose the speed advantage, but I don't really see why that
matters more on the guest side than the host.  Yes, fuzzing the guest
side matters more, but most guest side operations will induce passt to
perform a host side operation, so the speed of the host side handling
matters even if the guest side is what we care about fuzzing.

> That's something we already established a while ago when AbdAlRahman
> was working on it. We hadn't really looked into the host side yet, back
> then.
> 
> So, host side: we can't do it (and it's much less important) because we
> need to use those sockets in the same way passt uses them.

I'm not sure what you mean.  I outlined a way to do this below.

> The guest side interface is a trivial recv(), the host side is something
> complicated with iovecs and everything.

I'm now not sure if you're saying this in relation to approach B, or
approach A.

> > (B) is quite easy to do guest side - we just connect a "test guest" to
> > passt's socket.  Approach B is much harder for host side.  The current
> > draft has a test server listening on a single address.
> 
> This is just to get something up and running though, it obviously needs
> to be changed later.

I don't see how we are "up and running" if fuzzed packets from the
guest induce passt to forward them to random host side addresses that
we're not controlling - we won't generate reproducible results.

> > But that means
> > the fuzzing is fundamentally incapable of finding bugs involving
> > talking to multiple peers at once.  Worse, we have to constrain the
> > construction of guest side data so that we talk to the test server not
> > something else, and that's one of the things we most want to fuzz.
> > 
> > To really take approach B for the host side we'd need to intercept
> > *all* host side network traffic regardless of address.  Probably
> > easiest way to do that would be put the whole thing inside another
> > netns.  That outside netns would have a default route to a tap device,
> > and on the other side of the tap device would be a test server serving
> > up frames built from the fuzzer output.
> 
> As I was mentioning, this could be done in a network namespace without
> any interface, by making the test server listen to all ports and all
> addresses, with a non-local bind and a so-called AnyIP route. Tested:
> 
> $ pasta -- sh -c 'ip route add local default dev lo; nc -l 1 & { sleep 1; echo x | nc -N 1.2.3.4 1; }'
> x

True, but having the fuzzer synthesize L2 frames seems easier to me
than having it directly synthesize the various socket operations the
host side peer might perform.

> > It's probably easier to co-ordinate if the host side and guest side
> > test server is the same, so we'd have:
> > 
> > ,-------.                         ,-------------.
> > | AFL++ |-- shares memory with ---|  test peer  |
> > '-------'                         '--v------v---'
> >                                      |      |
> >                         /-tap device-/      |
> >                         |                   |
> > /- test netns ----------^-------------------|-------------\
> > |                                           |             |
> > |    ,--------.                             |             |
> > |    | passt  >-----------<unix socket>-----/             |
> > |    '--------'                                           |
> > \---------------------------------------------------------/
> > 
> > The test peer generates host side frames via the tap device, and guest
> > side frames via the Unix socket.
> 
> The UNIX socket is something we want to avoid, it's really much slower
> compared to shared memory (we tried something like that) on the path
> where AFL++ is trying to mutate data fast (because it can hit a lot of
> different code paths with small changes, compared to changing
> socket-side payload).

Right.  That's why I conclude approch A is probably better further
down.  But even if we only care about fuzzing guest side, we'll
usually incur the cost of operations on both sides, so I don't see
that using shared memory is more important for guest side than host
side interposition.

> > It could also be done with pasta, like this:
> > 
> > ,-------.                         ,-------------.
> > | AFL++ |-- shares memory with ---|  test peer  |
> > '-------'                         '--v------v---'
> >                                      |      |
> >                         /-tap device-/      packet
> >                         |                   socket
> > /- test netns ----------^-------------------|-------------\
> > |                                           |             |
> > |    ,--------.            ,----------------|---.         |
> > |    | pasta  >-tap device-< guest netns    *   |         |
> > |    '--------'            '--------------------'         |
> > \---------------------------------------------------------/
> > 
> > 
> > The order it generates host vs. guest frames should also come from the
> > fuzzer, not be fixed.  At least theoretically, this is non-invasive:
> > it could run with an unmodified passt/pasta.  Except that
> >   - AFL would still need coverage feedback from passt/pasta
> >   - It wouldn't allow us to simulate odd timings (except by actually
> >      expending real time)
> >   - The various interposing layers will probably slow down fuzzing.
> >   
> > So, I rather suspect it will work better to go fully to approach A: no
> > test peer at all, instead passt itself is modified to use mocked
> > versions of all the external syscalls to slurp data from AFL.  The
> > draft series already does this for epoll_wait() and recv*(), but we'd
> > need to also do that for recv*() on the tap socket, connect(),
> > accept(), TCP_INFO and probably others.  We'd also need to mock
> > "sending" calls, send(), write() and shutdown() at least - but those
> > could probably be no-ops.
> 
> ...except that by mocking all those we lose a lot of complexity where
> historically we had a ton of bugs. If we just mock recv() it's much
> less (well yes we had bugs there as well but it was like 3 or 4 over the
> entire project history).

If we mock as close as possible to the syscall level, I don't see that
bypass much of our complexity.  To be clear, I'm suggesting mocks
where both returned data and error codes are derived from the fuzzer,
not just no-op stubs.

> > This is more invasive, of course.  It also means we need to deal with
> > the case where AFL generates a syscall results that should be
> > impossible - that should move onto the next case ASAP, but not be
> > flagged as a passt bug.  On the other hand, this approach should be
> > fast, and since we can also mock clock_gettime(), the fuzzer can
> > potentially find timer logic bugs that would only occur after hours or
> > days in real time.
> >
> > > As far as I understood, it's not trivial to make the same instance
> > > of AFL++ share memory with two processes at the same time, so the
> > > memory-sharing path might need to take a more complicated turn, for
> > > example there could be a wrapper starting both passt and the test
> > > server and sharing memory with them, or passt could _additionally_
> > > (using a special out-of-band fuzzing channel) share data from AFL++
> > > with the test server.
> > > 
> > > An example of communication below (but events don't necessarily need
> > > to be in this order, this is just an example). For simplicity, let's
> > > ignore the fact that AFL++ might not directly share memory with passt
> > > and test server, and assume there are three areas of memory that
> > > AFL++ directly controls:
> > > 
> > > a. shared with passt: an array of struct epoll_event, 'ev'
> > > 
> > > b. shared with passt: the kind of tap-side buffer you implemented in
> > >    4/5, 'buf'
> > > 
> > > c. shared with the test server: a separate buffer, 'test_buf'
> > > 
> > > Example:
> > > 
> > > 1. AFL++ writes an EPOLLIN event in 'ev' with type
> > >    EPOLL_TYPE_TAP_PASST, of some data in 'buf', and some data in
> > >    'test_buf'
> > > 
> > > 2. AFL++ starts passt and the test server
> > > 
> > > 3. passt reads the EPOLL_TYPE_TAP_PASST event from 'ev', reads data
> > >    from 'buf' and hands it to passt_tap_handler()
> > > 
> > > 4. this happens to be have Ethernet, IP, and TCP headers, with the
> > >    SYN flag set, and destination address set to the address of the
> > >    test server (we might want to force all this, at least initially,
> > >    or give it as a hint to AFL++ somehow), so passt connects to
> > >    the test server
> > > 
> > > 5. the test server accepts the connection, and sends the contents
> > >    of 'test_buf' on it (for the test server, this is directly
> > >    payload, without headers, as they don't make sense there). I'm
> > >    not sure if we should have a different set of events (maybe we
> > >    need a "play script" for the server, in case?)
> > > 
> > > 6. this generates an EPOLLOUT event for passt. It's not in 'ev',
> > >    it's a regular epoll_wait() (I think we could have an
> > >    epoll_wait() loop where we additionally read one event from
> > >    'ev' for every iteration, or something like that)
> > > 
> > > 7. passt marks the connection as established and inserts it in the
> > >    flow table
> > > 
> > > 8. passt reads the data sent from the test server and generates
> > >    whatever TCP data packet to the "guest" (it might simply be
> > >    a sink)
> > > 
> > > ...and this attempt ends here because AFL++ generated a single
> > > event for passt, but there could be more (this should also be
> > > decided by AFL++).
> > > 
> > > Would something like this make sense?  
> 
> -- 
> Stefano
> 

-- 
David Gibson (he or they)	| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you, not the other way
				| around.
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]