[PATCH v2 00/22] RFC: Proof-of-concept based exeter+Avocado tests

[David Gibson <david@gibson.dropbear.id.au>]

Here's a rough proof of concept showing how we could run tests for
passt with Avocado and the exeter library I recently created.  It
includes Cleber's patch adding some basic Avocado tests and builds on
that.

The current draft is pretty janky:
 * The build rules to download and install the necessary pieces are messy
 * We create the Avocado job files from the exeter sources in the
   Makefile.  Ideally Avocado would eventually be extended to handle
   this itself
 * The names that Avocado sees for each test are overlong
 * There's some hacks to make sure things are executed from the
   right working directory

But, it's a starting point.

Stefano,

If you could look particularly at 6/22 and 22/22 which add the real
tests for passt/pasta, that would be great.  The more specific you can
be about what you find ugly about how the tests are written, then
better I can try to address that.

I suspect it will be easier to actually apply the series, then look at
the new test files (test/build/build.py, and test/pasta/pasta.py
particularly).  From there you can look at as much of the support
library as you need to, rather than digging through the actual patches
to look for that.

Cleber,

If you could look at 4..6/22 particularly to review how I'm connecting
the actual tests to the Avocado runner, that would be helpful.

Cleber Rosa (1):
  test: run static checkers with Avocado and JSON definitions

David Gibson (21):
  nstool: Fix some trivial typos
  nstool: Propagate SIGTERM to processes executed in the namespace
  test: Extend make targets to run Avocado tests
  test: Exeter based static tests
  test: Add exeter+Avocado based build tests
  test: Add linters for Python code
  tasst: Introduce library of common test helpers
  tasst: "snh" module for simulated network hosts
  tasst: Add helper to get network interface names for a site
  tasst: Add helpers to run commands with nstool
  tasst: Add ifup and network address helpers to SimNetHost
  tasst: Helper for creating veth devices between namespaces
  tasst: Add helper for getting MTU of a network interface
  tasst: Add helper to wait for IP address to appear
  tasst: Add helpers for getting a SimNetHost's routes
  tasst: Helpers to test transferring data between sites
  tasst: IP address allocation helpers
  tasst: Helpers for testing NDP behaviour
  tasst: Helpers for testing DHCP & DHCPv6 behaviour
  tasst: Helpers to construct a simple network environment for tests
  avocado: Convert basic pasta tests

 test/.gitignore                    |   2 +
 test/Makefile                      |  63 ++++++-
 test/avocado/static_checkers.json  |  16 ++
 test/build/.gitignore              |   2 +
 test/build/build.py                | 105 +++++++++++
 test/build/static_checkers.sh      |  28 +++
 test/meta/.gitignore               |   1 +
 test/meta/lint.sh                  |  28 +++
 test/nstool.c                      |  30 ++-
 test/pasta/.gitignore              |   1 +
 test/pasta/pasta.py                | 138 ++++++++++++++
 test/run_avocado                   |  51 ++++++
 test/tasst/.gitignore              |   1 +
 test/tasst/__init__.py             |  11 ++
 test/tasst/__main__.py             |  22 +++
 test/tasst/address.py              |  79 ++++++++
 test/tasst/dhcp.py                 | 132 ++++++++++++++
 test/tasst/dhcpv6.py               |  89 +++++++++
 test/tasst/ndp.py                  | 116 ++++++++++++
 test/tasst/nstool.py               | 186 +++++++++++++++++++
 test/tasst/pasta.py                |  52 ++++++
 test/tasst/scenario/__init__.py    |  12 ++
 test/tasst/scenario/simple.py      | 109 +++++++++++
 test/tasst/selftest/__init__.py    |  16 ++
 test/tasst/selftest/static_ifup.py |  60 ++++++
 test/tasst/selftest/veth.py        | 106 +++++++++++
 test/tasst/snh.py                  | 283 +++++++++++++++++++++++++++++
 test/tasst/transfer.py             | 194 ++++++++++++++++++++
 28 files changed, 1928 insertions(+), 5 deletions(-)
 create mode 100644 test/avocado/static_checkers.json
 create mode 100644 test/build/.gitignore
 create mode 100644 test/build/build.py
 create mode 100644 test/build/static_checkers.sh
 create mode 100644 test/meta/.gitignore
 create mode 100644 test/meta/lint.sh
 create mode 100644 test/pasta/.gitignore
 create mode 100644 test/pasta/pasta.py
 create mode 100755 test/run_avocado
 create mode 100644 test/tasst/.gitignore
 create mode 100644 test/tasst/__init__.py
 create mode 100644 test/tasst/__main__.py
 create mode 100644 test/tasst/address.py
 create mode 100644 test/tasst/dhcp.py
 create mode 100644 test/tasst/dhcpv6.py
 create mode 100644 test/tasst/ndp.py
 create mode 100644 test/tasst/nstool.py
 create mode 100644 test/tasst/pasta.py
 create mode 100644 test/tasst/scenario/__init__.py
 create mode 100644 test/tasst/scenario/simple.py
 create mode 100644 test/tasst/selftest/__init__.py
 create mode 100644 test/tasst/selftest/static_ifup.py
 create mode 100644 test/tasst/selftest/veth.py
 create mode 100644 test/tasst/snh.py
 create mode 100644 test/tasst/transfer.py

-- 
2.45.2