On Fri, Aug 14, 2026 at 09:35:59AM +0200, Stefano Brivio wrote: > On Fri, 14 Aug 2026 15:40:35 +1000 > David Gibson wrote: > > > On Thu, Aug 13, 2026 at 09:53:24AM +0200, Stefano Brivio wrote: > > > On Wed, 12 Aug 2026 12:56:28 +0530 > > > Anshu Kumari wrote: > > > > > > > Add fuzz-server that acts as passt's network peer > > > > during fuzzing. > > > > > > To me, this part makes sense. But this one: > > > > > > > It connects to passt's UNIX socket > > > > > > much less, while: > > > > > > > and listens on 127.0.0.1:9999 for TCP connections. > > > > > > this is the part that I expected instead. Otherwise it's not just > > > passt's network peer, it's the guest as well. > > > > > > > UNIX socket path: responds to ARP requests and TCP SYNs with > > > > stateless replies (swapped addresses, fixed ISN). Responses > > > > are XOR'd with AFL++ shared memory data so the fuzzer can > > > > mutate server behavior. > > > > > > This looks rather complicated to me. > > > > It does. > > > > > The approach I was suggesting with a test server is the following: > > > > > > > > > ,- exchanges guest-side data with ------------. > > > | ,---------|---------. > > > | ,--| passt | > > > | / '-.---------------^-' > > > ,---|---. / | connect(), | accept(), > > > | AFL++ |-- shares memory with ---| | send data, | reply with > > > '---|---' \ | etc. | data, etc. > > > | \ ,-v---------------'-. > > > | '--| test server | > > > | '---------|---------' > > > '- exchanges host-side data with -------------' > > > > > > ...at least in its basic form. Eventually, the test server should be > > > able to connect to passt itself (and we could call it "test peer" at > > > that point). > > > > So, I agree that to meaningfully fuzz things, we want the fuzzer to be > > able to control data on both the guest and host side. I can see two > > basic approaches: > > > > A) Alter passt/pasta so that instead of directly communicating with > > external entities (either guest or host side) we use mocked > > versions which retrieve data from AFL. We can do that either at > > the system call level, or at a higher helper function level, the > > lower level we go, the more of the "normal" passt code we're > > exercising, but doing at a slightly higher level might be easier > > to implement > > > > B) Run passt/pasta in an environment where we can intercept the > > external transfers to a test server / test peer / test guest which > > in turn responds based on data from AFL. > > > > Both the current draft and the sketch diagram Stefano has provided are > > a hybrid of both approaches, so far I'm not seeing a clear advantage > > to that over going all one way or the other. > > Of course, B) would be cleaner (and not that complicated, see below), > but we don't want to do that guest-side (sending data from a test > guest) because we would lose the speed advantage of having shared > memory on the path that _really_ matters for fuzzing (the guest is > untrusted, the kernel isn't). I agree we lose the speed advantage, but I don't really see why that matters more on the guest side than the host. Yes, fuzzing the guest side matters more, but most guest side operations will induce passt to perform a host side operation, so the speed of the host side handling matters even if the guest side is what we care about fuzzing. > That's something we already established a while ago when AbdAlRahman > was working on it. We hadn't really looked into the host side yet, back > then. > > So, host side: we can't do it (and it's much less important) because we > need to use those sockets in the same way passt uses them. I'm not sure what you mean. I outlined a way to do this below. > The guest side interface is a trivial recv(), the host side is something > complicated with iovecs and everything. I'm now not sure if you're saying this in relation to approach B, or approach A. > > (B) is quite easy to do guest side - we just connect a "test guest" to > > passt's socket. Approach B is much harder for host side. The current > > draft has a test server listening on a single address. > > This is just to get something up and running though, it obviously needs > to be changed later. I don't see how we are "up and running" if fuzzed packets from the guest induce passt to forward them to random host side addresses that we're not controlling - we won't generate reproducible results. > > But that means > > the fuzzing is fundamentally incapable of finding bugs involving > > talking to multiple peers at once. Worse, we have to constrain the > > construction of guest side data so that we talk to the test server not > > something else, and that's one of the things we most want to fuzz. > > > > To really take approach B for the host side we'd need to intercept > > *all* host side network traffic regardless of address. Probably > > easiest way to do that would be put the whole thing inside another > > netns. That outside netns would have a default route to a tap device, > > and on the other side of the tap device would be a test server serving > > up frames built from the fuzzer output. > > As I was mentioning, this could be done in a network namespace without > any interface, by making the test server listen to all ports and all > addresses, with a non-local bind and a so-called AnyIP route. Tested: > > $ pasta -- sh -c 'ip route add local default dev lo; nc -l 1 & { sleep 1; echo x | nc -N 1.2.3.4 1; }' > x True, but having the fuzzer synthesize L2 frames seems easier to me than having it directly synthesize the various socket operations the host side peer might perform. > > It's probably easier to co-ordinate if the host side and guest side > > test server is the same, so we'd have: > > > > ,-------. ,-------------. > > | AFL++ |-- shares memory with ---| test peer | > > '-------' '--v------v---' > > | | > > /-tap device-/ | > > | | > > /- test netns ----------^-------------------|-------------\ > > | | | > > | ,--------. | | > > | | passt >----------------/ | > > | '--------' | > > \---------------------------------------------------------/ > > > > The test peer generates host side frames via the tap device, and guest > > side frames via the Unix socket. > > The UNIX socket is something we want to avoid, it's really much slower > compared to shared memory (we tried something like that) on the path > where AFL++ is trying to mutate data fast (because it can hit a lot of > different code paths with small changes, compared to changing > socket-side payload). Right. That's why I conclude approch A is probably better further down. But even if we only care about fuzzing guest side, we'll usually incur the cost of operations on both sides, so I don't see that using shared memory is more important for guest side than host side interposition. > > It could also be done with pasta, like this: > > > > ,-------. ,-------------. > > | AFL++ |-- shares memory with ---| test peer | > > '-------' '--v------v---' > > | | > > /-tap device-/ packet > > | socket > > /- test netns ----------^-------------------|-------------\ > > | | | > > | ,--------. ,----------------|---. | > > | | pasta >-tap device-< guest netns * | | > > | '--------' '--------------------' | > > \---------------------------------------------------------/ > > > > > > The order it generates host vs. guest frames should also come from the > > fuzzer, not be fixed. At least theoretically, this is non-invasive: > > it could run with an unmodified passt/pasta. Except that > > - AFL would still need coverage feedback from passt/pasta > > - It wouldn't allow us to simulate odd timings (except by actually > > expending real time) > > - The various interposing layers will probably slow down fuzzing. > > > > So, I rather suspect it will work better to go fully to approach A: no > > test peer at all, instead passt itself is modified to use mocked > > versions of all the external syscalls to slurp data from AFL. The > > draft series already does this for epoll_wait() and recv*(), but we'd > > need to also do that for recv*() on the tap socket, connect(), > > accept(), TCP_INFO and probably others. We'd also need to mock > > "sending" calls, send(), write() and shutdown() at least - but those > > could probably be no-ops. > > ...except that by mocking all those we lose a lot of complexity where > historically we had a ton of bugs. If we just mock recv() it's much > less (well yes we had bugs there as well but it was like 3 or 4 over the > entire project history). If we mock as close as possible to the syscall level, I don't see that bypass much of our complexity. To be clear, I'm suggesting mocks where both returned data and error codes are derived from the fuzzer, not just no-op stubs. > > This is more invasive, of course. It also means we need to deal with > > the case where AFL generates a syscall results that should be > > impossible - that should move onto the next case ASAP, but not be > > flagged as a passt bug. On the other hand, this approach should be > > fast, and since we can also mock clock_gettime(), the fuzzer can > > potentially find timer logic bugs that would only occur after hours or > > days in real time. > > > > > As far as I understood, it's not trivial to make the same instance > > > of AFL++ share memory with two processes at the same time, so the > > > memory-sharing path might need to take a more complicated turn, for > > > example there could be a wrapper starting both passt and the test > > > server and sharing memory with them, or passt could _additionally_ > > > (using a special out-of-band fuzzing channel) share data from AFL++ > > > with the test server. > > > > > > An example of communication below (but events don't necessarily need > > > to be in this order, this is just an example). For simplicity, let's > > > ignore the fact that AFL++ might not directly share memory with passt > > > and test server, and assume there are three areas of memory that > > > AFL++ directly controls: > > > > > > a. shared with passt: an array of struct epoll_event, 'ev' > > > > > > b. shared with passt: the kind of tap-side buffer you implemented in > > > 4/5, 'buf' > > > > > > c. shared with the test server: a separate buffer, 'test_buf' > > > > > > Example: > > > > > > 1. AFL++ writes an EPOLLIN event in 'ev' with type > > > EPOLL_TYPE_TAP_PASST, of some data in 'buf', and some data in > > > 'test_buf' > > > > > > 2. AFL++ starts passt and the test server > > > > > > 3. passt reads the EPOLL_TYPE_TAP_PASST event from 'ev', reads data > > > from 'buf' and hands it to passt_tap_handler() > > > > > > 4. this happens to be have Ethernet, IP, and TCP headers, with the > > > SYN flag set, and destination address set to the address of the > > > test server (we might want to force all this, at least initially, > > > or give it as a hint to AFL++ somehow), so passt connects to > > > the test server > > > > > > 5. the test server accepts the connection, and sends the contents > > > of 'test_buf' on it (for the test server, this is directly > > > payload, without headers, as they don't make sense there). I'm > > > not sure if we should have a different set of events (maybe we > > > need a "play script" for the server, in case?) > > > > > > 6. this generates an EPOLLOUT event for passt. It's not in 'ev', > > > it's a regular epoll_wait() (I think we could have an > > > epoll_wait() loop where we additionally read one event from > > > 'ev' for every iteration, or something like that) > > > > > > 7. passt marks the connection as established and inserts it in the > > > flow table > > > > > > 8. passt reads the data sent from the test server and generates > > > whatever TCP data packet to the "guest" (it might simply be > > > a sink) > > > > > > ...and this attempt ends here because AFL++ generated a single > > > event for passt, but there could be more (this should also be > > > decided by AFL++). > > > > > > Would something like this make sense? > > -- > Stefano > -- David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson