From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by passt.top (Postfix) with ESMTPS id E96DB5A004E for ; Fri, 19 Jul 2024 08:59:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202312; t=1721372349; bh=Ah3FNjzGc30wVgw0kHZIfbTUNE0Qv9GW+OIqfuDVOX8=; h=From:To:Cc:Subject:Date:From; b=Y8+DKOeYT+bpFcBTeEFpRWmNdE4OVTNntfHpw3tFANQH03cZcjB4GmuuZY1NyDxm0 CHEliwn6pxiOhZ0LaJH65KsJjrtc+atoJA7ADI6uZlbsyABLGjAxWu1FBMUzldB53R ta47+0mRhN82ja8trNMET/abOsor5lkRjZT5f6+xebtl+st3WS7zSatY6EJLyCstr9 zuZ3zu+x9skd3K8MyDR984e6ggILWvk7EagExvQlLDF5Huv8h3t4oXRIV64jVQ87cR XJO3eaatiYZAFqnGdzCl0MhhjN5Si/24oFftd+r2DptItkZzjaGpcRaOz+W/Z7+OOB DpUHDGYLnq6Uw== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4WQLBP2DFfz4x4c; Fri, 19 Jul 2024 16:59:09 +1000 (AEST) From: David Gibson To: Stefano Brivio , passt-dev@passt.top Subject: [PATCH 0/4] RFC: Proof-of-concept based exeter+Avocado tests Date: Fri, 19 Jul 2024 16:59:03 +1000 Message-ID: <20240719065907.3539211-1-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.45.2 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-ID-Hash: PSFVRM2XBKWKBOXEFCLCHFOIRUVMRBTY X-Message-ID-Hash: PSFVRM2XBKWKBOXEFCLCHFOIRUVMRBTY X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Cleber Rosa , David Gibson X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Here's a rough proof of concept showing how we could run tests for passt with Avocado and the exeter library I recently created. It includes Cleber's patch adding some basic Avocado tests and builds on that. The current draft is pretty janky: * The build rules to download and install the necessary pieces are messy * We create the Avocado job files from the exeter sources in the Makefile. Ideally Avocado would eventually be extended to handle this itself * We have more ugly make rules to mangle the right venv path into the exeter based python script * The names that Avocado sees for each test are terrible (nondescriptive and duplicated). This is the point I made in an earlier reply to Cleber's patch - we need some way of setting the test name from the job files. * There's some hacks to make sure things are executed from the right working directory But, it's a starting point. Stefano, I'm hoping most of the jank in the setup can be cleaned up. The bit I really want your opinion on is how the test scripts themslves look: test/build/static_checkers.exeter (shell) and test/build/build.exeter.in (Python). Cleber, I hope this concrete example makes it a bit clearer how I'm thinking about running tests from Avocado, without making the test themselves inherently tied to Avocado. Cleber Rosa (1): test: run static checkers with Avocado and JSON definitions David Gibson (3): test: Extend make targets to run Avocado tests test: Exeter based static tests test: Add exeter+Avocado based build tests test/.gitignore | 2 + test/Makefile | 33 +++++++++++- test/avocado/static_checkers.json | 16 ++++++ test/build/.gitignore | 2 + test/build/build.exeter.in | 89 +++++++++++++++++++++++++++++++ test/build/static_checkers.exeter | 17 ++++++ test/run_avocado | 49 +++++++++++++++++ 7 files changed, 207 insertions(+), 1 deletion(-) create mode 100644 test/avocado/static_checkers.json create mode 100644 test/build/.gitignore create mode 100644 test/build/build.exeter.in create mode 100755 test/build/static_checkers.exeter create mode 100755 test/run_avocado -- 2.45.2