From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by passt.top (Postfix) with ESMTP id 0FFBB5A005E for ; Tue, 29 Nov 2022 14:44:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669729495; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=WJNGTyo0aiq3GbKswDu9mHw4r9VjR+xE1YS2F06tiXg=; b=KnlzJ7pMVxGyfoe73NW1i14/nbOaCRMN374ZmQFYiHHMZTHYz5wa+SMUZPC3XAOnt6+Uxn lrxSPFS6ZoGmEj5EW4cxsEI3LubG8axIiBGSmh6LyJXd2ehG0dAn4WlbTbph5KfvMRPxDR 3LDDvzcPzZ+ABYRTP7u2Ny7dT5kMuRY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-316-xYu9fMDXPqm4iz5610Bobw-1; Tue, 29 Nov 2022 08:44:54 -0500 X-MC-Unique: xYu9fMDXPqm4iz5610Bobw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E52B885A5A6 for ; Tue, 29 Nov 2022 13:44:53 +0000 (UTC) Received: from localhost (unknown [10.39.194.2]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8854EC15BA4; Tue, 29 Nov 2022 13:44:53 +0000 (UTC) Date: Tue, 29 Nov 2022 13:44:50 +0000 From: "Richard W.M. Jones" To: Stefano Brivio Subject: Re: [PATCH passt v2 0/7] Add fuzzing Message-ID: <20221129134450.GZ7636@redhat.com> References: <20221117184938.2270462-1-rjones@redhat.com> <20221125102354.0540ad95@elisabeth> <20221125101103.GO7636@redhat.com> <20221129143442.4ae32a9b@elisabeth> MIME-Version: 1.0 In-Reply-To: <20221129143442.4ae32a9b@elisabeth> User-Agent: Mutt/1.5.21 (2010-09-15) X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Message-ID-Hash: L4YDGOXXSWQ6HBMDOWWR7HNWLLJOAIWD X-Message-ID-Hash: L4YDGOXXSWQ6HBMDOWWR7HNWLLJOAIWD X-MailFrom: rjones@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.3 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Tue, Nov 29, 2022 at 02:34:42PM +0100, Stefano Brivio wrote: > On Fri, 25 Nov 2022 10:11:03 +0000 > "Richard W.M. Jones" wrote: > > > On Fri, Nov 25, 2022 at 10:23:54AM +0100, Stefano Brivio wrote: > > > and introducing frames with special values, as you hinted on IRC, for > > > example one-byte frames with commands such as "go ahead with socket > > > processing then come back to 'tap' frames", so that passt has a chance > > > to do some meaningful socket-side operations before getting back to > > > fuzz input. > > > > You can improve the chance that the fuzzer will find these frames by > > either including them in test cases (we need better test cases, which > > is separate issue), or by making the encoding such that they are easy > > to find. eg. if you had four possible values, encode them only in the > > bottom two bits and ignore the higher bits. Since these frames are > > only used for fuzzing you can change the meaning of them later, so > > exact encoding isn't an ABI issue. > > Right, yes, I was thinking we could, under #ifdef FUZZING, accept > frames that are shorter than a 802.3 header (up to 13 bytes), and take > their length (in network order, but I guess AFL++ can easily get > familiar with it) as fuzzing flow commands. > > About test cases, I'm not sure this should be included in regular test > runs, because there's no reasonable definition for a test duration. I'd > rather have a separate script which keeps running indefinitely, updating > sources as they become available. Yes for libnbd/nbdkit/hivex we don't (can't) fuzz as part of regular tests. It's a separate process. You might be able to apply to this programme: https://google.github.io/oss-fuzz/ (You'll still need to add the fuzzing support upstream.) > > > Patch 7/7 is very useful and appreciated anyway as it demystifies the > > > whole topic for me, and we can probably recycle most of the > > > documentation. I'm not sure yet how/if the wrapper still fits with the > > > stuff I'm looking into. > > > > It would definitely be better to have passt itself be able to > > read a file off disk. > > > > For example when we fuzz nbdkit we do not used or need a wrapper, > > because nbdkit has an -s / --single option that reads from stdin and > > writes to stdout. This was originally added to inetd support. > > > > We drive nbdkit from the fuzzer directly like this: > > > > afl-fuzz -i fuzzing/testcase_dir -o fuzzing/sync_dir -M fuzz01 \ > > ./server/nbdkit -s -t 1 ./plugins/memory/.libs/nbdkit-memory-plugin.so 1M > > > > (https://gitlab.com/nbdkit/nbdkit/-/blob/ef035f7090d8bec2700ef1f941e371d351d647ad/fuzzing/README#L35-36) > > Ah, I didn't know, interesting. On the other hand, would it really make > sense to add support for that (which has probably no use other than > fuzzing) instead of a hack with __AFL_FUZZ_TESTCASE_BUF and mmap()? > > According to AFL++ documentation that should speed things up > considerably: > https://github.com/AFLplusplus/AFLplusplus/blob/stable/instrumentation/README.persistent_mode.md#5-shared-memory-fuzzing Yes if you can get __AFL_FUZZ_TESTCASE_BUF working that would be even better. For nbdkit it seemed like it would be quite difficult because we would need to ensure that the binary can be completely "reset" between runs without being re-exec'd, so we'd need to chase down every possible allocation / initialization and make sure it's reversed. We get pretty good fuzzing performance by the method above so it was never a priority. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top