From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by passt.top (Postfix) with ESMTPS id 780B45A026E for ; Thu, 20 Apr 2023 03:11:25 +0200 (CEST) Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4Q203Y5Vthz4xDr; Thu, 20 Apr 2023 11:11:21 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=201602; t=1681953081; bh=f9c8mFV5XGDCOT3fTLBdY/SlVUSYHIhOFtfmX9Pxgfk=; h=From:To:Cc:Subject:Date:From; b=CcWIzdqAnmvY3ijMXo/8F50YzXd3D9oq6npXxADOg92gx3JSZ8zhUCO+1sHjNiJc9 MH245NhLlmoRsi3ydCzpIGZzfU+mPFzQcBI7VOH+7i8Yi+KdLMTpns8ZjS/3q00bnH 0G6xO5kcQrPcoIWFoVTLzD6DX/Pt1aevPVgTzZT8= From: David Gibson To: passt-dev@passt.top, Stefano Brivio Subject: [PATCH 0/7] RFC: Proof-of-concept conversion of some tests to Avocado framework Date: Thu, 20 Apr 2023 11:11:01 +1000 Message-Id: <20230420011108.494181-1-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.40.0 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-ID-Hash: FECMX6O7N53QB2CGYZGKV4S7GK44EP7B X-Message-ID-Hash: FECMX6O7N53QB2CGYZGKV4S7GK44EP7B X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: David Gibson X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: As discussed on calls, I've been investigating Avocado as a possible replacement for our hand-rolled shell test framework. This is a proof-of-concept conversion of a few of the tests to the Avocado framework. Specifically, the static checkers, the build tests and the pasta transfer tests are converted. I think it's a promising start. On my laptop running the Avocado suite takes less than half the time of the equivalent tests in the old framework (1m10 vs 2m30), despite the fact that the avocado tests are reconstructing their test environment for every single test. There's also a handful of additional tests in Avocado (mostly for the test mechanics themselves). David Gibson (7): avocado: Make a duplicate copy of testsuite for comparison purposes avocado: Don't double download assets for test/ and oldtest/ avocado: Move static checkers to avocado avocado: Convert build tests to avocado avocado: Helper functions and classes to run commands via nstool avocado: Helper to get link-local address and wait for SLAAC to complete avocado: Convert pasta transfer tests to Avocado Makefile | 11 + avocado/.gitignore | 1 + avocado/build.py | 83 +++ avocado/common.py | 213 +++++++ avocado/pasta.py | 196 ++++++ avocado/pasta_dhcp.py | 95 +++ avocado/pasta_ndp.py | 52 ++ avocado/pasta_tcp.py | 61 ++ avocado/pasta_udp.py | 55 ++ avocado/static_checkers/clang-tidy.sh | 3 + avocado/static_checkers/cppcheck.sh | 3 + oldtest/.gitignore | 11 + oldtest/Makefile | 119 ++++ oldtest/README.md | 137 +++++ {test => oldtest}/build/all | 0 {test => oldtest}/build/clang_tidy | 0 {test => oldtest}/build/cppcheck | 0 oldtest/ci | 1 + oldtest/demo/passt | 245 ++++++++ oldtest/demo/pasta | 274 +++++++++ oldtest/demo/podman | 819 ++++++++++++++++++++++++++ oldtest/distro/debian | 252 ++++++++ oldtest/distro/fedora | 396 +++++++++++++ oldtest/distro/opensuse | 208 +++++++ oldtest/distro/ubuntu | 216 +++++++ oldtest/env/mate-terminal.profile | 42 ++ oldtest/find-arm64-firmware.sh | 13 + oldtest/lib/context | 130 ++++ oldtest/lib/layout | 259 ++++++++ oldtest/lib/layout_ugly | 113 ++++ oldtest/lib/perf_report | 272 +++++++++ oldtest/lib/setup | 385 ++++++++++++ oldtest/lib/setup_ugly | 58 ++ oldtest/lib/term | 750 +++++++++++++++++++++++ oldtest/lib/test | 398 +++++++++++++ oldtest/lib/util | 133 +++++ oldtest/lib/video | 152 +++++ oldtest/memory/passt | 187 ++++++ oldtest/nstool.c | 565 ++++++++++++++++++ oldtest/passt.mbuto | 83 +++ oldtest/passt.mem.mbuto | 44 ++ oldtest/passt/dhcp | 70 +++ oldtest/passt/ndp | 33 ++ oldtest/passt/shutdown | 19 + oldtest/passt/tcp | 76 +++ oldtest/passt/udp | 46 ++ oldtest/passt_in_ns/icmp | 32 + oldtest/passt_in_ns/shutdown | 19 + oldtest/passt_in_ns/tcp | 256 ++++++++ oldtest/passt_in_ns/udp | 138 +++++ {test => oldtest}/pasta/dhcp | 0 {test => oldtest}/pasta/ndp | 0 {test => oldtest}/pasta/tcp | 0 {test => oldtest}/pasta/udp | 0 oldtest/pasta_options/log_to_file | 93 +++ oldtest/perf/passt_tcp | 215 +++++++ oldtest/perf/passt_udp | 165 ++++++ oldtest/perf/pasta_tcp | 300 ++++++++++ oldtest/perf/pasta_udp | 219 +++++++ oldtest/prepare-distro-img.sh | 18 + oldtest/run | 238 ++++++++ oldtest/run_demo | 1 + oldtest/two_guests/basic | 80 +++ oldtest/valgrind.supp | 9 + test/lib/layout | 31 - test/lib/setup | 37 -- test/run | 14 - 67 files changed, 9062 insertions(+), 82 deletions(-) create mode 100644 avocado/.gitignore create mode 100644 avocado/build.py create mode 100644 avocado/common.py create mode 100644 avocado/pasta.py create mode 100644 avocado/pasta_dhcp.py create mode 100644 avocado/pasta_ndp.py create mode 100644 avocado/pasta_tcp.py create mode 100644 avocado/pasta_udp.py create mode 100755 avocado/static_checkers/clang-tidy.sh create mode 100755 avocado/static_checkers/cppcheck.sh create mode 100644 oldtest/.gitignore create mode 100644 oldtest/Makefile create mode 100644 oldtest/README.md rename {test => oldtest}/build/all (100%) rename {test => oldtest}/build/clang_tidy (100%) rename {test => oldtest}/build/cppcheck (100%) create mode 120000 oldtest/ci create mode 100644 oldtest/demo/passt create mode 100644 oldtest/demo/pasta create mode 100644 oldtest/demo/podman create mode 100644 oldtest/distro/debian create mode 100644 oldtest/distro/fedora create mode 100644 oldtest/distro/opensuse create mode 100644 oldtest/distro/ubuntu create mode 100644 oldtest/env/mate-terminal.profile create mode 100755 oldtest/find-arm64-firmware.sh create mode 100644 oldtest/lib/context create mode 100644 oldtest/lib/layout create mode 100644 oldtest/lib/layout_ugly create mode 100755 oldtest/lib/perf_report create mode 100755 oldtest/lib/setup create mode 100755 oldtest/lib/setup_ugly create mode 100755 oldtest/lib/term create mode 100755 oldtest/lib/test create mode 100755 oldtest/lib/util create mode 100755 oldtest/lib/video create mode 100644 oldtest/memory/passt create mode 100644 oldtest/nstool.c create mode 100755 oldtest/passt.mbuto create mode 100755 oldtest/passt.mem.mbuto create mode 100644 oldtest/passt/dhcp create mode 100644 oldtest/passt/ndp create mode 100644 oldtest/passt/shutdown create mode 100644 oldtest/passt/tcp create mode 100644 oldtest/passt/udp create mode 100644 oldtest/passt_in_ns/icmp create mode 100644 oldtest/passt_in_ns/shutdown create mode 100644 oldtest/passt_in_ns/tcp create mode 100644 oldtest/passt_in_ns/udp rename {test => oldtest}/pasta/dhcp (100%) rename {test => oldtest}/pasta/ndp (100%) rename {test => oldtest}/pasta/tcp (100%) rename {test => oldtest}/pasta/udp (100%) create mode 100644 oldtest/pasta_options/log_to_file create mode 100644 oldtest/perf/passt_tcp create mode 100644 oldtest/perf/passt_udp create mode 100644 oldtest/perf/pasta_tcp create mode 100644 oldtest/perf/pasta_udp create mode 100755 oldtest/prepare-distro-img.sh create mode 100755 oldtest/run create mode 120000 oldtest/run_demo create mode 100644 oldtest/two_guests/basic create mode 100644 oldtest/valgrind.supp -- 2.40.0