public inbox for passt-dev@passt.top
 help / color / mirror / code / Atom feed
* [PATCH 0/7] RFC: Proof-of-concept conversion of some tests to Avocado framework
@ 2023-04-20  1:11 David Gibson
  2023-04-20  1:11 ` [PATCH 1/7] avocado: Make a duplicate copy of testsuite for comparison purposes David Gibson
                   ` (6 more replies)
  0 siblings, 7 replies; 8+ messages in thread
From: David Gibson @ 2023-04-20  1:11 UTC (permalink / raw)
  To: passt-dev, Stefano Brivio; +Cc: David Gibson

As discussed on calls, I've been investigating Avocado as a possible
replacement for our hand-rolled shell test framework.  This is a
proof-of-concept conversion of a few of the tests to the Avocado
framework.  Specifically, the static checkers, the build tests and the
pasta transfer tests are converted.

I think it's a promising start.  On my laptop running the Avocado
suite takes less than half the time of the equivalent tests in the old
framework (1m10 vs 2m30), despite the fact that the avocado tests are
reconstructing their test environment for every single test.  There's
also a handful of additional tests in Avocado (mostly for the test
mechanics themselves).

David Gibson (7):
  avocado: Make a duplicate copy of testsuite for comparison purposes
  avocado: Don't double download assets for test/ and oldtest/
  avocado: Move static checkers to avocado
  avocado: Convert build tests to avocado
  avocado: Helper functions and classes to run commands via nstool
  avocado: Helper to get link-local address and wait for SLAAC to
    complete
  avocado: Convert pasta transfer tests to Avocado

 Makefile                              |  11 +
 avocado/.gitignore                    |   1 +
 avocado/build.py                      |  83 +++
 avocado/common.py                     | 213 +++++++
 avocado/pasta.py                      | 196 ++++++
 avocado/pasta_dhcp.py                 |  95 +++
 avocado/pasta_ndp.py                  |  52 ++
 avocado/pasta_tcp.py                  |  61 ++
 avocado/pasta_udp.py                  |  55 ++
 avocado/static_checkers/clang-tidy.sh |   3 +
 avocado/static_checkers/cppcheck.sh   |   3 +
 oldtest/.gitignore                    |  11 +
 oldtest/Makefile                      | 119 ++++
 oldtest/README.md                     | 137 +++++
 {test => oldtest}/build/all           |   0
 {test => oldtest}/build/clang_tidy    |   0
 {test => oldtest}/build/cppcheck      |   0
 oldtest/ci                            |   1 +
 oldtest/demo/passt                    | 245 ++++++++
 oldtest/demo/pasta                    | 274 +++++++++
 oldtest/demo/podman                   | 819 ++++++++++++++++++++++++++
 oldtest/distro/debian                 | 252 ++++++++
 oldtest/distro/fedora                 | 396 +++++++++++++
 oldtest/distro/opensuse               | 208 +++++++
 oldtest/distro/ubuntu                 | 216 +++++++
 oldtest/env/mate-terminal.profile     |  42 ++
 oldtest/find-arm64-firmware.sh        |  13 +
 oldtest/lib/context                   | 130 ++++
 oldtest/lib/layout                    | 259 ++++++++
 oldtest/lib/layout_ugly               | 113 ++++
 oldtest/lib/perf_report               | 272 +++++++++
 oldtest/lib/setup                     | 385 ++++++++++++
 oldtest/lib/setup_ugly                |  58 ++
 oldtest/lib/term                      | 750 +++++++++++++++++++++++
 oldtest/lib/test                      | 398 +++++++++++++
 oldtest/lib/util                      | 133 +++++
 oldtest/lib/video                     | 152 +++++
 oldtest/memory/passt                  | 187 ++++++
 oldtest/nstool.c                      | 565 ++++++++++++++++++
 oldtest/passt.mbuto                   |  83 +++
 oldtest/passt.mem.mbuto               |  44 ++
 oldtest/passt/dhcp                    |  70 +++
 oldtest/passt/ndp                     |  33 ++
 oldtest/passt/shutdown                |  19 +
 oldtest/passt/tcp                     |  76 +++
 oldtest/passt/udp                     |  46 ++
 oldtest/passt_in_ns/icmp              |  32 +
 oldtest/passt_in_ns/shutdown          |  19 +
 oldtest/passt_in_ns/tcp               | 256 ++++++++
 oldtest/passt_in_ns/udp               | 138 +++++
 {test => oldtest}/pasta/dhcp          |   0
 {test => oldtest}/pasta/ndp           |   0
 {test => oldtest}/pasta/tcp           |   0
 {test => oldtest}/pasta/udp           |   0
 oldtest/pasta_options/log_to_file     |  93 +++
 oldtest/perf/passt_tcp                | 215 +++++++
 oldtest/perf/passt_udp                | 165 ++++++
 oldtest/perf/pasta_tcp                | 300 ++++++++++
 oldtest/perf/pasta_udp                | 219 +++++++
 oldtest/prepare-distro-img.sh         |  18 +
 oldtest/run                           | 238 ++++++++
 oldtest/run_demo                      |   1 +
 oldtest/two_guests/basic              |  80 +++
 oldtest/valgrind.supp                 |   9 +
 test/lib/layout                       |  31 -
 test/lib/setup                        |  37 --
 test/run                              |  14 -
 67 files changed, 9062 insertions(+), 82 deletions(-)
 create mode 100644 avocado/.gitignore
 create mode 100644 avocado/build.py
 create mode 100644 avocado/common.py
 create mode 100644 avocado/pasta.py
 create mode 100644 avocado/pasta_dhcp.py
 create mode 100644 avocado/pasta_ndp.py
 create mode 100644 avocado/pasta_tcp.py
 create mode 100644 avocado/pasta_udp.py
 create mode 100755 avocado/static_checkers/clang-tidy.sh
 create mode 100755 avocado/static_checkers/cppcheck.sh
 create mode 100644 oldtest/.gitignore
 create mode 100644 oldtest/Makefile
 create mode 100644 oldtest/README.md
 rename {test => oldtest}/build/all (100%)
 rename {test => oldtest}/build/clang_tidy (100%)
 rename {test => oldtest}/build/cppcheck (100%)
 create mode 120000 oldtest/ci
 create mode 100644 oldtest/demo/passt
 create mode 100644 oldtest/demo/pasta
 create mode 100644 oldtest/demo/podman
 create mode 100644 oldtest/distro/debian
 create mode 100644 oldtest/distro/fedora
 create mode 100644 oldtest/distro/opensuse
 create mode 100644 oldtest/distro/ubuntu
 create mode 100644 oldtest/env/mate-terminal.profile
 create mode 100755 oldtest/find-arm64-firmware.sh
 create mode 100644 oldtest/lib/context
 create mode 100644 oldtest/lib/layout
 create mode 100644 oldtest/lib/layout_ugly
 create mode 100755 oldtest/lib/perf_report
 create mode 100755 oldtest/lib/setup
 create mode 100755 oldtest/lib/setup_ugly
 create mode 100755 oldtest/lib/term
 create mode 100755 oldtest/lib/test
 create mode 100755 oldtest/lib/util
 create mode 100755 oldtest/lib/video
 create mode 100644 oldtest/memory/passt
 create mode 100644 oldtest/nstool.c
 create mode 100755 oldtest/passt.mbuto
 create mode 100755 oldtest/passt.mem.mbuto
 create mode 100644 oldtest/passt/dhcp
 create mode 100644 oldtest/passt/ndp
 create mode 100644 oldtest/passt/shutdown
 create mode 100644 oldtest/passt/tcp
 create mode 100644 oldtest/passt/udp
 create mode 100644 oldtest/passt_in_ns/icmp
 create mode 100644 oldtest/passt_in_ns/shutdown
 create mode 100644 oldtest/passt_in_ns/tcp
 create mode 100644 oldtest/passt_in_ns/udp
 rename {test => oldtest}/pasta/dhcp (100%)
 rename {test => oldtest}/pasta/ndp (100%)
 rename {test => oldtest}/pasta/tcp (100%)
 rename {test => oldtest}/pasta/udp (100%)
 create mode 100644 oldtest/pasta_options/log_to_file
 create mode 100644 oldtest/perf/passt_tcp
 create mode 100644 oldtest/perf/passt_udp
 create mode 100644 oldtest/perf/pasta_tcp
 create mode 100644 oldtest/perf/pasta_udp
 create mode 100755 oldtest/prepare-distro-img.sh
 create mode 100755 oldtest/run
 create mode 120000 oldtest/run_demo
 create mode 100644 oldtest/two_guests/basic
 create mode 100644 oldtest/valgrind.supp

-- 
2.40.0


^ permalink raw reply	[flat|nested] 8+ messages in thread
end of thread, other threads:[~2023-04-20  1:11 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-04-20  1:11 [PATCH 0/7] RFC: Proof-of-concept conversion of some tests to Avocado framework David Gibson
2023-04-20  1:11 ` [PATCH 1/7] avocado: Make a duplicate copy of testsuite for comparison purposes David Gibson
2023-04-20  1:11 ` [PATCH 2/7] avocado: Don't double download assets for test/ and oldtest/ David Gibson
2023-04-20  1:11 ` [PATCH 3/7] avocado: Move static checkers to avocado David Gibson
2023-04-20  1:11 ` [PATCH 4/7] avocado: Convert build tests " David Gibson
2023-04-20  1:11 ` [PATCH 5/7] avocado: Helper functions and classes to run commands via nstool David Gibson
2023-04-20  1:11 ` [PATCH 6/7] avocado: Helper to get link-local address and wait for SLAAC to complete David Gibson
2023-04-20  1:11 ` [PATCH 7/7] avocado: Convert pasta transfer tests to Avocado David Gibson

Code repositories for project(s) associated with this public inbox

	https://passt.top/passt

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for IMAP folder(s).