Re: [PATCH passt v2 0/7] Add fuzzing

[Stefano Brivio <sbrivio@redhat.com>]

On Thu, 17 Nov 2022 18:49:31 +0000
"Richard W.M. Jones" <rjones@redhat.com> wrote:

> With this series, fuzzing actually works, albeit slowly.  More on that
> below.
> 
> Patches 1 & 2 are the same as before.
> 
> Patch 3 is Stefano Brivio's modified patch (with some changes that we
> discussed together on IRC but otherwise unchanged).
> 
> Patch 4 is new, but discussed already upstream: It changes the order
> in which EPOLLIN and EPOLLRDHUP events are processed, so that we don't
> drop packets when the socket is closed.
>
> Patches 5 & 6 are the hacks that were needed to get fuzzing to work.
> Patch 6 removes all seccomp and other isolation stuff because for
> unclear reasons that breaks AFL instrumentation.  AFL appears to fork
> off a second process, and somehow strace cannot follow that process,
> but the second process fails, and that breaks AFL completely.  Without
> strace data it's rather hard to see what's going on so I didn't
> investigate this further.
> 
> Patch 7 adds the fuzzing wrapper and is not greatly changed from
> before.  However I did have to disable the AFL "fork server"
> optimization which somehow doesn't work with passt (it does work fine
> with libnbd & nbdkit).
> 
> Speed is not great.  I'm getting ~ 75-80 execs/second.  Really we want
> this to be much higher, since that ultimately governs how fast we can
> explore new code paths and find bugs.  Ideally well over 1000 execs/s
> (per fuzzing process) would be a good target to aim for.  (Of course
> this depends on the hardware as well.)

Applied up to 4/7, thanks!

For the rest: I have a local branch with 5/7 and 6/7 fixed up: the
'fuzzing' Makefile target enables the syscalls you listed and avoids
prctl(PR_SET_DUMPABLE, 0) in isolation_postfork() via -DFUZZING.

I managed to speed things up by skipping some operations not needed
for fuzzing, but just a tiny bit (~200 execs/s). I'm looking into
switching to persistent mode:
  https://github.com/AFLplusplus/AFLplusplus/blob/stable/instrumentation/README.persistent_mode.md

and introducing frames with special values, as you hinted on IRC, for
example one-byte frames with commands such as "go ahead with socket
processing then come back to 'tap' frames", so that passt has a chance
to do some meaningful socket-side operations before getting back to
fuzz input.

Patch 7/7 is very useful and appreciated anyway as it demystifies the
whole topic for me, and we can probably recycle most of the
documentation. I'm not sure yet how/if the wrapper still fits with the
stuff I'm looking into.

-- 
Stefano