From: David Gibson <david@gibson.dropbear.id.au>
To: passt-dev@passt.top, Stefano Brivio <sbrivio@redhat.com>
Cc: crosa@redhat.com, jarichte@redhat.com,
David Gibson <david@gibson.dropbear.id.au>
Subject: [PATCH 00/27] RFC: Start converting passt & pasta tests to Avocado using special plugin
Date: Tue, 27 Jun 2023 12:54:01 +1000 [thread overview]
Message-ID: <20230627025429.2209702-1-david@gibson.dropbear.id.au> (raw)
I've sent previous series converting some of the passt & pasta
testsuite to use Avocado. However the jUnit-like structure of avocado
tests wasn't a great fit for our needs (plus Stefano and I just
dislike the style). Here's another attempt, where instead we use a
new "avocado-classless" plugin to avoid jUnit-ism.
This is definitely a work-in-progress. Some early observations:
* It is noticeably less verbose than jUnit style
* It does avoid some of the confusing stuff that arises from jUnit style
* The present draft has some different pretty confusing stuff,
involving nested and stacked function decorators, and some other
non-entirely-obvious use of higher order functions
* I hope I can improve this at least a bit, but it will need some thought
* I was pleasantly surprised at how simple it was to write the
plugin itself
David Gibson (27):
avocado: Make a duplicate copy of testsuite for comparison purposes
avocado: Don't double download assets for test/ and oldtest/
avocado: Move static checkers to avocado
avocado: Introduce "avocado-classless" plugin, runner and outline
avocado, test: Add static checkers for Python code
avocado: Resolver implementation for avocado-classless plugin
avocado: Add basic assertion helpers to avocado-classless plugin
tasst, avocado: Introduce library of common test helpers
avocado-classless: Test matrices by composition
tasst: Helper functions for executing commands in different places
avocado-classless: Allow overriding default timeout
avocado: Convert build tests to avocado
tasst: Add helpers for running background commands on sites
tasst: Add helper to get network interface names for a site
tasst: Add helpers to run commands with nstool
tasst: Add ifup and network address helpers to Site
tasst: Helper for creating veth devices between namespaces
tasst: Add helper for getting MTU of a network interface
tasst: Add helper to wait for IP address to appear
tasst: Add helpers for getting a site's routes
tasst: Helpers to test transferring data between sites
tasst: IP address allocation helpers
tasst: Helpers for running daemons with a pidfile
tasst: Helpers for testing NDP behaviour
tasst: Helpers for testing DHCP & DHCPv6 behaviour
tasst: Helpers to construct a simple network environment for tests
avocado: Convert basic pasta tests
Makefile | 9 +
oldtest/.gitignore | 11 +
oldtest/Makefile | 119 +++
oldtest/README.md | 137 +++
{test => oldtest}/build/all | 0
{test => oldtest}/build/clang_tidy | 0
{test => oldtest}/build/cppcheck | 0
oldtest/ci | 1 +
oldtest/demo/passt | 245 ++++++
oldtest/demo/pasta | 274 ++++++
oldtest/demo/podman | 819 ++++++++++++++++++
oldtest/distro/debian | 252 ++++++
oldtest/distro/fedora | 396 +++++++++
oldtest/distro/opensuse | 208 +++++
oldtest/distro/ubuntu | 216 +++++
oldtest/env/mate-terminal.profile | 42 +
oldtest/find-arm64-firmware.sh | 13 +
oldtest/lib/context | 130 +++
oldtest/lib/layout | 259 ++++++
oldtest/lib/layout_ugly | 113 +++
oldtest/lib/perf_report | 272 ++++++
oldtest/lib/setup | 385 ++++++++
oldtest/lib/setup_ugly | 58 ++
oldtest/lib/term | 750 ++++++++++++++++
oldtest/lib/test | 398 +++++++++
oldtest/lib/util | 133 +++
oldtest/lib/video | 152 ++++
oldtest/memory/passt | 187 ++++
oldtest/nstool.c | 565 ++++++++++++
oldtest/passt.mbuto | 83 ++
oldtest/passt.mem.mbuto | 44 +
oldtest/passt/dhcp | 70 ++
oldtest/passt/ndp | 33 +
oldtest/passt/shutdown | 19 +
oldtest/passt/tcp | 76 ++
oldtest/passt/udp | 46 +
oldtest/passt_in_ns/icmp | 32 +
oldtest/passt_in_ns/shutdown | 19 +
oldtest/passt_in_ns/tcp | 256 ++++++
oldtest/passt_in_ns/udp | 138 +++
{test => oldtest}/pasta/dhcp | 0
{test => oldtest}/pasta/ndp | 0
{test => oldtest}/pasta/tcp | 0
{test => oldtest}/pasta/udp | 0
oldtest/pasta_options/log_to_file | 93 ++
oldtest/perf/passt_tcp | 215 +++++
oldtest/perf/passt_udp | 165 ++++
oldtest/perf/pasta_tcp | 300 +++++++
oldtest/perf/pasta_udp | 219 +++++
oldtest/prepare-distro-img.sh | 18 +
oldtest/run | 238 +++++
oldtest/run_demo | 1 +
oldtest/two_guests/basic | 80 ++
oldtest/valgrind.supp | 9 +
test/.gitignore | 2 +
test/Makefile | 69 +-
test/avocado/build.py | 94 ++
test/avocado/pasta.py | 129 +++
test/avocado/static_checkers/clang-tidy.sh | 3 +
test/avocado/static_checkers/cppcheck.sh | 3 +
test/avocado/static_checkers/flake8.sh | 3 +
test/avocado/static_checkers/pylint.sh | 3 +
test/avocado_classless/.gitignore | 1 +
.../avocado_classless/__init__.py | 11 +
.../avocado_classless/manifest.py | 48 +
.../avocado_classless/plugin.py | 231 +++++
.../avocado_classless/test.py | 80 ++
test/avocado_classless/examples.py | 57 ++
test/avocado_classless/selftests.py | 76 ++
test/avocado_classless/setup.py | 32 +
test/lib/layout | 31 -
test/lib/setup | 47 -
test/run | 14 -
test/tasst/__init__.py | 11 +
test/tasst/address.py | 80 ++
test/tasst/dhcp.py | 114 +++
test/tasst/dhcpv6.py | 74 ++
test/tasst/exesite.py | 285 ++++++
test/tasst/meta/__init__.py | 16 +
test/tasst/meta/static_ifup.py | 60 ++
test/tasst/meta/veth.py | 86 ++
test/tasst/ndp.py | 99 +++
test/tasst/nstool.py | 184 ++++
test/tasst/pasta.py | 42 +
test/tasst/scenario/__init__.py | 12 +
test/tasst/scenario/simple.py | 98 +++
test/tasst/transfer.py | 174 ++++
test/tasst/typecheck.py | 47 +
88 files changed, 10518 insertions(+), 96 deletions(-)
create mode 100644 oldtest/.gitignore
create mode 100644 oldtest/Makefile
create mode 100644 oldtest/README.md
rename {test => oldtest}/build/all (100%)
rename {test => oldtest}/build/clang_tidy (100%)
rename {test => oldtest}/build/cppcheck (100%)
create mode 120000 oldtest/ci
create mode 100644 oldtest/demo/passt
create mode 100644 oldtest/demo/pasta
create mode 100644 oldtest/demo/podman
create mode 100644 oldtest/distro/debian
create mode 100644 oldtest/distro/fedora
create mode 100644 oldtest/distro/opensuse
create mode 100644 oldtest/distro/ubuntu
create mode 100644 oldtest/env/mate-terminal.profile
create mode 100755 oldtest/find-arm64-firmware.sh
create mode 100644 oldtest/lib/context
create mode 100644 oldtest/lib/layout
create mode 100644 oldtest/lib/layout_ugly
create mode 100755 oldtest/lib/perf_report
create mode 100755 oldtest/lib/setup
create mode 100755 oldtest/lib/setup_ugly
create mode 100755 oldtest/lib/term
create mode 100755 oldtest/lib/test
create mode 100755 oldtest/lib/util
create mode 100755 oldtest/lib/video
create mode 100644 oldtest/memory/passt
create mode 100644 oldtest/nstool.c
create mode 100755 oldtest/passt.mbuto
create mode 100755 oldtest/passt.mem.mbuto
create mode 100644 oldtest/passt/dhcp
create mode 100644 oldtest/passt/ndp
create mode 100644 oldtest/passt/shutdown
create mode 100644 oldtest/passt/tcp
create mode 100644 oldtest/passt/udp
create mode 100644 oldtest/passt_in_ns/icmp
create mode 100644 oldtest/passt_in_ns/shutdown
create mode 100644 oldtest/passt_in_ns/tcp
create mode 100644 oldtest/passt_in_ns/udp
rename {test => oldtest}/pasta/dhcp (100%)
rename {test => oldtest}/pasta/ndp (100%)
rename {test => oldtest}/pasta/tcp (100%)
rename {test => oldtest}/pasta/udp (100%)
create mode 100644 oldtest/pasta_options/log_to_file
create mode 100644 oldtest/perf/passt_tcp
create mode 100644 oldtest/perf/passt_udp
create mode 100644 oldtest/perf/pasta_tcp
create mode 100644 oldtest/perf/pasta_udp
create mode 100755 oldtest/prepare-distro-img.sh
create mode 100755 oldtest/run
create mode 120000 oldtest/run_demo
create mode 100644 oldtest/two_guests/basic
create mode 100644 oldtest/valgrind.supp
create mode 100644 test/avocado/build.py
create mode 100644 test/avocado/pasta.py
create mode 100755 test/avocado/static_checkers/clang-tidy.sh
create mode 100755 test/avocado/static_checkers/cppcheck.sh
create mode 100755 test/avocado/static_checkers/flake8.sh
create mode 100755 test/avocado/static_checkers/pylint.sh
create mode 100644 test/avocado_classless/.gitignore
create mode 100644 test/avocado_classless/avocado_classless/__init__.py
create mode 100644 test/avocado_classless/avocado_classless/manifest.py
create mode 100644 test/avocado_classless/avocado_classless/plugin.py
create mode 100644 test/avocado_classless/avocado_classless/test.py
create mode 100644 test/avocado_classless/examples.py
create mode 100644 test/avocado_classless/selftests.py
create mode 100644 test/avocado_classless/setup.py
create mode 100644 test/tasst/__init__.py
create mode 100644 test/tasst/address.py
create mode 100644 test/tasst/dhcp.py
create mode 100644 test/tasst/dhcpv6.py
create mode 100644 test/tasst/exesite.py
create mode 100644 test/tasst/meta/__init__.py
create mode 100644 test/tasst/meta/static_ifup.py
create mode 100644 test/tasst/meta/veth.py
create mode 100644 test/tasst/ndp.py
create mode 100644 test/tasst/nstool.py
create mode 100644 test/tasst/pasta.py
create mode 100644 test/tasst/scenario/__init__.py
create mode 100644 test/tasst/scenario/simple.py
create mode 100644 test/tasst/transfer.py
create mode 100644 test/tasst/typecheck.py
--
2.41.0
next reply other threads:[~2023-06-27 2:54 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-06-27 2:54 David Gibson [this message]
2023-06-27 2:54 ` [PATCH 01/27] avocado: Make a duplicate copy of testsuite for comparison purposes David Gibson
2023-06-27 2:54 ` [PATCH 02/27] avocado: Don't double download assets for test/ and oldtest/ David Gibson
2023-06-27 2:54 ` [PATCH 03/27] avocado: Move static checkers to avocado David Gibson
2023-06-27 2:54 ` [PATCH 04/27] avocado: Introduce "avocado-classless" plugin, runner and outline David Gibson
2023-06-27 2:54 ` [PATCH 05/27] avocado, test: Add static checkers for Python code David Gibson
2023-06-27 2:54 ` [PATCH 06/27] avocado: Resolver implementation for avocado-classless plugin David Gibson
2023-06-27 2:54 ` [PATCH 07/27] avocado: Add basic assertion helpers to " David Gibson
2023-06-27 2:54 ` [PATCH 08/27] tasst, avocado: Introduce library of common test helpers David Gibson
2023-06-27 2:54 ` [PATCH 09/27] avocado-classless: Test matrices by composition David Gibson
2023-06-27 2:54 ` [PATCH 10/27] tasst: Helper functions for executing commands in different places David Gibson
2023-06-27 2:54 ` [PATCH 11/27] avocado-classless: Allow overriding default timeout David Gibson
2023-06-27 2:54 ` [PATCH 12/27] avocado: Convert build tests to avocado David Gibson
2023-06-27 2:54 ` [PATCH 13/27] tasst: Add helpers for running background commands on sites David Gibson
2023-06-27 2:54 ` [PATCH 14/27] tasst: Add helper to get network interface names for a site David Gibson
2023-06-27 2:54 ` [PATCH 15/27] tasst: Add helpers to run commands with nstool David Gibson
2023-06-27 2:54 ` [PATCH 16/27] tasst: Add ifup and network address helpers to Site David Gibson
2023-06-27 2:54 ` [PATCH 17/27] tasst: Helper for creating veth devices between namespaces David Gibson
2023-06-27 2:54 ` [PATCH 18/27] tasst: Add helper for getting MTU of a network interface David Gibson
2023-06-27 2:54 ` [PATCH 19/27] tasst: Add helper to wait for IP address to appear David Gibson
2023-06-27 2:54 ` [PATCH 20/27] tasst: Add helpers for getting a site's routes David Gibson
2023-06-27 2:54 ` [PATCH 21/27] tasst: Helpers to test transferring data between sites David Gibson
2023-06-27 2:54 ` [PATCH 22/27] tasst: IP address allocation helpers David Gibson
2023-06-27 2:54 ` [PATCH 23/27] tasst: Helpers for running daemons with a pidfile David Gibson
2023-06-27 2:54 ` [PATCH 24/27] tasst: Helpers for testing NDP behaviour David Gibson
2023-06-27 2:54 ` [PATCH 25/27] tasst: Helpers for testing DHCP & DHCPv6 behaviour David Gibson
2023-06-27 2:54 ` [PATCH 26/27] tasst: Helpers to construct a simple network environment for tests David Gibson
2023-06-27 2:54 ` [PATCH 27/27] avocado: Convert basic pasta tests David Gibson
2023-07-05 0:30 ` Stefano Brivio
2023-07-05 3:27 ` David Gibson
2023-07-07 17:42 ` Stefano Brivio
2023-07-10 7:45 ` David Gibson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230627025429.2209702-1-david@gibson.dropbear.id.au \
--to=david@gibson.dropbear.id.au \
--cc=crosa@redhat.com \
--cc=jarichte@redhat.com \
--cc=passt-dev@passt.top \
--cc=sbrivio@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://passt.top/passt
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for IMAP folder(s).