Re: [PATCH passt v2 0/7] Add fuzzing

["Richard W.M. Jones" <rjones@redhat.com>]

On Tue, Nov 29, 2022 at 02:34:42PM +0100, Stefano Brivio wrote:
> On Fri, 25 Nov 2022 10:11:03 +0000
> "Richard W.M. Jones" <rjones@redhat.com> wrote:
> 
> > On Fri, Nov 25, 2022 at 10:23:54AM +0100, Stefano Brivio wrote:
> > > and introducing frames with special values, as you hinted on IRC, for
> > > example one-byte frames with commands such as "go ahead with socket
> > > processing then come back to 'tap' frames", so that passt has a chance
> > > to do some meaningful socket-side operations before getting back to
> > > fuzz input.  
> > 
> > You can improve the chance that the fuzzer will find these frames by
> > either including them in test cases (we need better test cases, which
> > is separate issue), or by making the encoding such that they are easy
> > to find.  eg. if you had four possible values, encode them only in the
> > bottom two bits and ignore the higher bits.  Since these frames are
> > only used for fuzzing you can change the meaning of them later, so
> > exact encoding isn't an ABI issue.
> 
> Right, yes, I was thinking we could, under #ifdef FUZZING, accept
> frames that are shorter than a 802.3 header (up to 13 bytes), and take
> their length (in network order, but I guess AFL++ can easily get
> familiar with it) as fuzzing flow commands.
> 
> About test cases, I'm not sure this should be included in regular test
> runs, because there's no reasonable definition for a test duration. I'd
> rather have a separate script which keeps running indefinitely, updating
> sources as they become available.

Yes for libnbd/nbdkit/hivex we don't (can't) fuzz as part of regular tests.
It's a separate process.

You might be able to apply to this programme:

https://google.github.io/oss-fuzz/

(You'll still need to add the fuzzing support upstream.)

> > > Patch 7/7 is very useful and appreciated anyway as it demystifies the
> > > whole topic for me, and we can probably recycle most of the
> > > documentation. I'm not sure yet how/if the wrapper still fits with the
> > > stuff I'm looking into.  
> > 
> > It would definitely be better to have passt itself be able to
> > read a file off disk.
> > 
> > For example when we fuzz nbdkit we do not used or need a wrapper,
> > because nbdkit has an -s / --single option that reads from stdin and
> > writes to stdout.  This was originally added to inetd support.
> > 
> > We drive nbdkit from the fuzzer directly like this:
> > 
> >   afl-fuzz -i fuzzing/testcase_dir -o fuzzing/sync_dir -M fuzz01 \
> >       ./server/nbdkit -s -t 1 ./plugins/memory/.libs/nbdkit-memory-plugin.so 1M
> > 
> > (https://gitlab.com/nbdkit/nbdkit/-/blob/ef035f7090d8bec2700ef1f941e371d351d647ad/fuzzing/README#L35-36)
> 
> Ah, I didn't know, interesting. On the other hand, would it really make
> sense to add support for that (which has probably no use other than
> fuzzing) instead of a hack with __AFL_FUZZ_TESTCASE_BUF and mmap()?
> 
> According to AFL++ documentation that should speed things up
> considerably:
>   https://github.com/AFLplusplus/AFLplusplus/blob/stable/instrumentation/README.persistent_mode.md#5-shared-memory-fuzzing

Yes if you can get __AFL_FUZZ_TESTCASE_BUF working that would be even
better.  For nbdkit it seemed like it would be quite difficult because
we would need to ensure that the binary can be completely "reset"
between runs without being re-exec'd, so we'd need to chase down every
possible allocation / initialization and make sure it's reversed.  We
get pretty good fuzzing performance by the method above so it was
never a priority.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top