public inbox for passt-dev@passt.top
 help / color / mirror / code / Atom feed
* [PATCH 00/27] RFC: Start converting passt & pasta tests to Avocado using special plugin
@ 2023-06-27  2:54 David Gibson
  2023-06-27  2:54 ` [PATCH 01/27] avocado: Make a duplicate copy of testsuite for comparison purposes David Gibson
                   ` (26 more replies)
  0 siblings, 27 replies; 32+ messages in thread
From: David Gibson @ 2023-06-27  2:54 UTC (permalink / raw)
  To: passt-dev, Stefano Brivio; +Cc: crosa, jarichte, David Gibson

I've sent previous series converting some of the passt & pasta
testsuite to use Avocado.  However the jUnit-like structure of avocado
tests wasn't a great fit for our needs (plus Stefano and I just
dislike the style).  Here's another attempt, where instead we use a
new "avocado-classless" plugin to avoid jUnit-ism.

This is definitely a work-in-progress.  Some early observations:

  * It is noticeably less verbose than jUnit style
  * It does avoid some of the confusing stuff that arises from jUnit style
  * The present draft has some different pretty confusing stuff,
    involving nested and stacked function decorators, and some other
    non-entirely-obvious use of higher order functions
     * I hope I can improve this at least a bit, but it will need some thought
  * I was pleasantly surprised at how simple it was to write the
    plugin itself

David Gibson (27):
  avocado: Make a duplicate copy of testsuite for comparison purposes
  avocado: Don't double download assets for test/ and oldtest/
  avocado: Move static checkers to avocado
  avocado: Introduce "avocado-classless" plugin, runner and outline
  avocado, test: Add static checkers for Python code
  avocado: Resolver implementation for avocado-classless plugin
  avocado: Add basic assertion helpers to avocado-classless plugin
  tasst, avocado: Introduce library of common test helpers
  avocado-classless: Test matrices by composition
  tasst: Helper functions for executing commands in different places
  avocado-classless: Allow overriding default timeout
  avocado: Convert build tests to avocado
  tasst: Add helpers for running background commands on sites
  tasst: Add helper to get network interface names for a site
  tasst: Add helpers to run commands with nstool
  tasst: Add ifup and network address helpers to Site
  tasst: Helper for creating veth devices between namespaces
  tasst: Add helper for getting MTU of a network interface
  tasst: Add helper to wait for IP address to appear
  tasst: Add helpers for getting a site's routes
  tasst: Helpers to test transferring data between sites
  tasst: IP address allocation helpers
  tasst: Helpers for running daemons with a pidfile
  tasst: Helpers for testing NDP behaviour
  tasst: Helpers for testing DHCP & DHCPv6 behaviour
  tasst: Helpers to construct a simple network environment for tests
  avocado: Convert basic pasta tests

 Makefile                                      |   9 +
 oldtest/.gitignore                            |  11 +
 oldtest/Makefile                              | 119 +++
 oldtest/README.md                             | 137 +++
 {test => oldtest}/build/all                   |   0
 {test => oldtest}/build/clang_tidy            |   0
 {test => oldtest}/build/cppcheck              |   0
 oldtest/ci                                    |   1 +
 oldtest/demo/passt                            | 245 ++++++
 oldtest/demo/pasta                            | 274 ++++++
 oldtest/demo/podman                           | 819 ++++++++++++++++++
 oldtest/distro/debian                         | 252 ++++++
 oldtest/distro/fedora                         | 396 +++++++++
 oldtest/distro/opensuse                       | 208 +++++
 oldtest/distro/ubuntu                         | 216 +++++
 oldtest/env/mate-terminal.profile             |  42 +
 oldtest/find-arm64-firmware.sh                |  13 +
 oldtest/lib/context                           | 130 +++
 oldtest/lib/layout                            | 259 ++++++
 oldtest/lib/layout_ugly                       | 113 +++
 oldtest/lib/perf_report                       | 272 ++++++
 oldtest/lib/setup                             | 385 ++++++++
 oldtest/lib/setup_ugly                        |  58 ++
 oldtest/lib/term                              | 750 ++++++++++++++++
 oldtest/lib/test                              | 398 +++++++++
 oldtest/lib/util                              | 133 +++
 oldtest/lib/video                             | 152 ++++
 oldtest/memory/passt                          | 187 ++++
 oldtest/nstool.c                              | 565 ++++++++++++
 oldtest/passt.mbuto                           |  83 ++
 oldtest/passt.mem.mbuto                       |  44 +
 oldtest/passt/dhcp                            |  70 ++
 oldtest/passt/ndp                             |  33 +
 oldtest/passt/shutdown                        |  19 +
 oldtest/passt/tcp                             |  76 ++
 oldtest/passt/udp                             |  46 +
 oldtest/passt_in_ns/icmp                      |  32 +
 oldtest/passt_in_ns/shutdown                  |  19 +
 oldtest/passt_in_ns/tcp                       | 256 ++++++
 oldtest/passt_in_ns/udp                       | 138 +++
 {test => oldtest}/pasta/dhcp                  |   0
 {test => oldtest}/pasta/ndp                   |   0
 {test => oldtest}/pasta/tcp                   |   0
 {test => oldtest}/pasta/udp                   |   0
 oldtest/pasta_options/log_to_file             |  93 ++
 oldtest/perf/passt_tcp                        | 215 +++++
 oldtest/perf/passt_udp                        | 165 ++++
 oldtest/perf/pasta_tcp                        | 300 +++++++
 oldtest/perf/pasta_udp                        | 219 +++++
 oldtest/prepare-distro-img.sh                 |  18 +
 oldtest/run                                   | 238 +++++
 oldtest/run_demo                              |   1 +
 oldtest/two_guests/basic                      |  80 ++
 oldtest/valgrind.supp                         |   9 +
 test/.gitignore                               |   2 +
 test/Makefile                                 |  69 +-
 test/avocado/build.py                         |  94 ++
 test/avocado/pasta.py                         | 129 +++
 test/avocado/static_checkers/clang-tidy.sh    |   3 +
 test/avocado/static_checkers/cppcheck.sh      |   3 +
 test/avocado/static_checkers/flake8.sh        |   3 +
 test/avocado/static_checkers/pylint.sh        |   3 +
 test/avocado_classless/.gitignore             |   1 +
 .../avocado_classless/__init__.py             |  11 +
 .../avocado_classless/manifest.py             |  48 +
 .../avocado_classless/plugin.py               | 231 +++++
 .../avocado_classless/test.py                 |  80 ++
 test/avocado_classless/examples.py            |  57 ++
 test/avocado_classless/selftests.py           |  76 ++
 test/avocado_classless/setup.py               |  32 +
 test/lib/layout                               |  31 -
 test/lib/setup                                |  47 -
 test/run                                      |  14 -
 test/tasst/__init__.py                        |  11 +
 test/tasst/address.py                         |  80 ++
 test/tasst/dhcp.py                            | 114 +++
 test/tasst/dhcpv6.py                          |  74 ++
 test/tasst/exesite.py                         | 285 ++++++
 test/tasst/meta/__init__.py                   |  16 +
 test/tasst/meta/static_ifup.py                |  60 ++
 test/tasst/meta/veth.py                       |  86 ++
 test/tasst/ndp.py                             |  99 +++
 test/tasst/nstool.py                          | 184 ++++
 test/tasst/pasta.py                           |  42 +
 test/tasst/scenario/__init__.py               |  12 +
 test/tasst/scenario/simple.py                 |  98 +++
 test/tasst/transfer.py                        | 174 ++++
 test/tasst/typecheck.py                       |  47 +
 88 files changed, 10518 insertions(+), 96 deletions(-)
 create mode 100644 oldtest/.gitignore
 create mode 100644 oldtest/Makefile
 create mode 100644 oldtest/README.md
 rename {test => oldtest}/build/all (100%)
 rename {test => oldtest}/build/clang_tidy (100%)
 rename {test => oldtest}/build/cppcheck (100%)
 create mode 120000 oldtest/ci
 create mode 100644 oldtest/demo/passt
 create mode 100644 oldtest/demo/pasta
 create mode 100644 oldtest/demo/podman
 create mode 100644 oldtest/distro/debian
 create mode 100644 oldtest/distro/fedora
 create mode 100644 oldtest/distro/opensuse
 create mode 100644 oldtest/distro/ubuntu
 create mode 100644 oldtest/env/mate-terminal.profile
 create mode 100755 oldtest/find-arm64-firmware.sh
 create mode 100644 oldtest/lib/context
 create mode 100644 oldtest/lib/layout
 create mode 100644 oldtest/lib/layout_ugly
 create mode 100755 oldtest/lib/perf_report
 create mode 100755 oldtest/lib/setup
 create mode 100755 oldtest/lib/setup_ugly
 create mode 100755 oldtest/lib/term
 create mode 100755 oldtest/lib/test
 create mode 100755 oldtest/lib/util
 create mode 100755 oldtest/lib/video
 create mode 100644 oldtest/memory/passt
 create mode 100644 oldtest/nstool.c
 create mode 100755 oldtest/passt.mbuto
 create mode 100755 oldtest/passt.mem.mbuto
 create mode 100644 oldtest/passt/dhcp
 create mode 100644 oldtest/passt/ndp
 create mode 100644 oldtest/passt/shutdown
 create mode 100644 oldtest/passt/tcp
 create mode 100644 oldtest/passt/udp
 create mode 100644 oldtest/passt_in_ns/icmp
 create mode 100644 oldtest/passt_in_ns/shutdown
 create mode 100644 oldtest/passt_in_ns/tcp
 create mode 100644 oldtest/passt_in_ns/udp
 rename {test => oldtest}/pasta/dhcp (100%)
 rename {test => oldtest}/pasta/ndp (100%)
 rename {test => oldtest}/pasta/tcp (100%)
 rename {test => oldtest}/pasta/udp (100%)
 create mode 100644 oldtest/pasta_options/log_to_file
 create mode 100644 oldtest/perf/passt_tcp
 create mode 100644 oldtest/perf/passt_udp
 create mode 100644 oldtest/perf/pasta_tcp
 create mode 100644 oldtest/perf/pasta_udp
 create mode 100755 oldtest/prepare-distro-img.sh
 create mode 100755 oldtest/run
 create mode 120000 oldtest/run_demo
 create mode 100644 oldtest/two_guests/basic
 create mode 100644 oldtest/valgrind.supp
 create mode 100644 test/avocado/build.py
 create mode 100644 test/avocado/pasta.py
 create mode 100755 test/avocado/static_checkers/clang-tidy.sh
 create mode 100755 test/avocado/static_checkers/cppcheck.sh
 create mode 100755 test/avocado/static_checkers/flake8.sh
 create mode 100755 test/avocado/static_checkers/pylint.sh
 create mode 100644 test/avocado_classless/.gitignore
 create mode 100644 test/avocado_classless/avocado_classless/__init__.py
 create mode 100644 test/avocado_classless/avocado_classless/manifest.py
 create mode 100644 test/avocado_classless/avocado_classless/plugin.py
 create mode 100644 test/avocado_classless/avocado_classless/test.py
 create mode 100644 test/avocado_classless/examples.py
 create mode 100644 test/avocado_classless/selftests.py
 create mode 100644 test/avocado_classless/setup.py
 create mode 100644 test/tasst/__init__.py
 create mode 100644 test/tasst/address.py
 create mode 100644 test/tasst/dhcp.py
 create mode 100644 test/tasst/dhcpv6.py
 create mode 100644 test/tasst/exesite.py
 create mode 100644 test/tasst/meta/__init__.py
 create mode 100644 test/tasst/meta/static_ifup.py
 create mode 100644 test/tasst/meta/veth.py
 create mode 100644 test/tasst/ndp.py
 create mode 100644 test/tasst/nstool.py
 create mode 100644 test/tasst/pasta.py
 create mode 100644 test/tasst/scenario/__init__.py
 create mode 100644 test/tasst/scenario/simple.py
 create mode 100644 test/tasst/transfer.py
 create mode 100644 test/tasst/typecheck.py

-- 
2.41.0


^ permalink raw reply	[flat|nested] 32+ messages in thread

end of thread, other threads:[~2023-07-11  8:04 UTC | newest]

Thread overview: 32+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-06-27  2:54 [PATCH 00/27] RFC: Start converting passt & pasta tests to Avocado using special plugin David Gibson
2023-06-27  2:54 ` [PATCH 01/27] avocado: Make a duplicate copy of testsuite for comparison purposes David Gibson
2023-06-27  2:54 ` [PATCH 02/27] avocado: Don't double download assets for test/ and oldtest/ David Gibson
2023-06-27  2:54 ` [PATCH 03/27] avocado: Move static checkers to avocado David Gibson
2023-06-27  2:54 ` [PATCH 04/27] avocado: Introduce "avocado-classless" plugin, runner and outline David Gibson
2023-06-27  2:54 ` [PATCH 05/27] avocado, test: Add static checkers for Python code David Gibson
2023-06-27  2:54 ` [PATCH 06/27] avocado: Resolver implementation for avocado-classless plugin David Gibson
2023-06-27  2:54 ` [PATCH 07/27] avocado: Add basic assertion helpers to " David Gibson
2023-06-27  2:54 ` [PATCH 08/27] tasst, avocado: Introduce library of common test helpers David Gibson
2023-06-27  2:54 ` [PATCH 09/27] avocado-classless: Test matrices by composition David Gibson
2023-06-27  2:54 ` [PATCH 10/27] tasst: Helper functions for executing commands in different places David Gibson
2023-06-27  2:54 ` [PATCH 11/27] avocado-classless: Allow overriding default timeout David Gibson
2023-06-27  2:54 ` [PATCH 12/27] avocado: Convert build tests to avocado David Gibson
2023-06-27  2:54 ` [PATCH 13/27] tasst: Add helpers for running background commands on sites David Gibson
2023-06-27  2:54 ` [PATCH 14/27] tasst: Add helper to get network interface names for a site David Gibson
2023-06-27  2:54 ` [PATCH 15/27] tasst: Add helpers to run commands with nstool David Gibson
2023-06-27  2:54 ` [PATCH 16/27] tasst: Add ifup and network address helpers to Site David Gibson
2023-06-27  2:54 ` [PATCH 17/27] tasst: Helper for creating veth devices between namespaces David Gibson
2023-06-27  2:54 ` [PATCH 18/27] tasst: Add helper for getting MTU of a network interface David Gibson
2023-06-27  2:54 ` [PATCH 19/27] tasst: Add helper to wait for IP address to appear David Gibson
2023-06-27  2:54 ` [PATCH 20/27] tasst: Add helpers for getting a site's routes David Gibson
2023-06-27  2:54 ` [PATCH 21/27] tasst: Helpers to test transferring data between sites David Gibson
2023-06-27  2:54 ` [PATCH 22/27] tasst: IP address allocation helpers David Gibson
2023-06-27  2:54 ` [PATCH 23/27] tasst: Helpers for running daemons with a pidfile David Gibson
2023-06-27  2:54 ` [PATCH 24/27] tasst: Helpers for testing NDP behaviour David Gibson
2023-06-27  2:54 ` [PATCH 25/27] tasst: Helpers for testing DHCP & DHCPv6 behaviour David Gibson
2023-06-27  2:54 ` [PATCH 26/27] tasst: Helpers to construct a simple network environment for tests David Gibson
2023-06-27  2:54 ` [PATCH 27/27] avocado: Convert basic pasta tests David Gibson
2023-07-05  0:30   ` Stefano Brivio
2023-07-05  3:27     ` David Gibson
2023-07-07 17:42       ` Stefano Brivio
2023-07-10  7:45         ` David Gibson

Code repositories for project(s) associated with this public inbox

	https://passt.top/passt

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for IMAP folder(s).