From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Gibson To: passt-dev@passt.top Subject: Re: [PATCH] test: Add rudimentary support to run selected tests only Date: Fri, 07 Oct 2022 17:11:56 +1100 Message-ID: In-Reply-To: <20221007004732.1188903-1-sbrivio@redhat.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5367386367043524075==" --===============5367386367043524075== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On Fri, Oct 07, 2022 at 02:47:32AM +0200, Stefano Brivio wrote: > To keep this simple, only support tests that have corresponding setup > and teardown functions implied by their path. For example: >=20 > ./run passt/ndp >=20 > will trigger the 'passt' setup and teardown functions. >=20 > This is not really elegant, but it looks robust, and while David is > considering proper alternatives, it should be quite useful. >=20 > Signed-off-by: Stefano Brivio I'm not sure I'd go so far as to call it "robust". AFAICT it will break for cases where the setup function isn't named after the test dir (e.g. for perf/* which need the passt_in_ns setup). And, of course, it's still the user's responsibility to handle inter-test dependencies (dhcp needs ndp to run first , tcp & udp need dhcp to run first , etc.) But still, it's pretty simple and looks useful despite those limitations so, Reviewed-by: David Gibson > --- > test/README.md | 16 ++++++++++++++++ > test/lib/term | 2 +- > test/run | 37 ++++++++++++++++++++++++++++++++++++- > 3 files changed, 53 insertions(+), 2 deletions(-) >=20 > diff --git a/test/README.md b/test/README.md > index 72a4986..09ad05d 100644 > --- a/test/README.md > +++ b/test/README.md > @@ -85,6 +85,22 @@ variable settings: DEBUG=3D1 enables debugging messages,= TRACE=3D1 enables tracing > =20 > PCAP=3D1 TRACE=3D1 ./run > =20 > +## Running selected tests > + > +Rudimentary support to run a list of selected tests, without support for > +dependencies, is available. Tests need to have a setup function correspond= ing to > +their path. For example: > + > + ./run passt/ndp passt/dhcp pasta/ndp > + > +will call the 'passt' setup function (from lib/setup), run the two corresp= onding > +tests, call the 'passt' teardown function, the 'pasta' setup, run the past= a/ndp > +test, and finally tear down the 'pasta' setup. > + > +Note that requirements on steps implemented by related tests are not handl= ed. > +For example, if the 'passt/tcp' needs guest connectivity set up by the > +'passt/ndp' and 'passt/dhcp' tests, those need to be listed explicitly. > + > ## Continuous integration > =20 > Issuing: > diff --git a/test/lib/term b/test/lib/term > index eade2cd..1b42df8 100755 > --- a/test/lib/term > +++ b/test/lib/term > @@ -650,7 +650,7 @@ run_term() { > asciinema rec --overwrite "${STATEBASE}/demo.uncut" -c "$TMUX /bin/sh -c= './run_demo from_term'" > video_postprocess "${STATEBASE}/demo.uncut" > else > - $TMUX /bin/sh -c './run from_term' > + $TMUX /bin/sh -c "./run from_term ${*}" > fi > } > =20 > diff --git a/test/run b/test/run > index 4bb9cd8..1ae270e 100755 > --- a/test/run > +++ b/test/run > @@ -127,6 +127,37 @@ run() { > return 0 > } > =20 > +# run_selected() - Run list of tests, with setup/teardown based on test pa= th > +# $@: List of tests > +run_selected() { > + mkfifo $STATEBASE/log_pipe > + > + term > + VALGRIND=3D1 > + > + __setup=3D > + for __test; do > + if [ "${__test%%/*}" !=3D "${__setup}" ]; then > + [ -n "${__setup}" ] && teardown "${__setup}" > + __setup=3D"${__test%%/*}" > + setup "${__setup}" > + fi > + > + test "${__test}" > + done > + teardown "${__setup}" > + > + log "PASS: ${STATUS_PASS}, FAIL: ${STATUS_FAIL}" > + > + pause_continue \ > + "Press any key to keep test session open" \ > + "Closing in " \ > + "Interrupted, press any key to quit" \ > + 9 > + > + return 0 > +} > + > # demo() - Simpler path for demo purposes > demo() { > mkfifo $STATEBASE/log_pipe > @@ -160,11 +191,15 @@ demo() { > [ "$(basename "${0}")" =3D "run_demo" ] && DEMO=3D1 > =20 > if [ "${1}" =3D "from_term" ]; then > + shift > + > exec > ${LOGDIR}/script.log 2>&1 > [ ${DEBUG} -eq 1 ] && set -x > cd .. > if [ ${DEMO} -eq 1 ]; then > demo > + elif [ -n "${1}" ]; then > + run_selected ${*} > else > run > fi > @@ -176,7 +211,7 @@ else > :> "${LOGFILE}" > STATEBASE=3D"$(mktemp -d --tmpdir passt-tests-XXXXXX)" > trap "cleanup" EXIT > - run_term > + run_term ${*} > fi > =20 > [ ${DEMO} -eq 1 ] && exit 0 --=20 David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson --===============5367386367043524075== Content-Type: application/pgp-signature Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="signature.asc" MIME-Version: 1.0 LS0tLS1CRUdJTiBQR1AgU0lHTkFUVVJFLS0tLS0KCmlRSXpCQUVCQ0FBZEZpRUVvVUx4V3U0L1dz MGRCK1h0Z3lwWTRnRXdZU0lGQW1NL3d5VUFDZ2tRZ3lwWTRnRXcKWVNMWVh4QUFtamFVTkJnVmlK c0ZCYzhSSjBvcFE0V2lSMjRSMUxpdTZxbDRTN0djazFlWWEyd2djbzVjYzVvMApDQUdNS2l2ZDdv VEV2NGdpc2dadXNKb0E2K3dJaEl6dGJyRHIzN1R6QWNRWUlmbnJmdmt5K2hGSHJXcXlRL2N4CjVz SHdQQWZCNERpL2tjK2lUZFBRSDZPMnVIMUs5dXJtdjBPeEVHUFZqbXVYTURzaDZ3cEZIK0Y3VmVq T3BWeFAKbU9vaFlab2UrVFZxYmZobVBuTkpKK2dGeXF5emlhNTdaQ3A3NW16M01iQWZNcnAyaFdi VnU2MEJ2aTViSk4zZAozeVNVT01XZUZNcG9zMVc4MnQwZm4zU3RuSGFFQklLWTZ0WlYxMXo0SC80 Q3NmWHZvUmRTMzM5bkJqK3Z2bDcrCjdoai8reW1nalJTTjc1cWMzMmN6NE9MbFFrVWVHTElBSi9j UFQ2N3pEUTNSZDRtSER2NXNnTVJOTXBLa1E4ZzYKNUphVG1oUGdXdjNMc24yTjRHckVyeU84ZjBo a1JIWnNJTHBmNkRJVnNvT3plSjFOVWwrWHFUaUZDWWswS1QrcAowQXhXRVdXNmkxa2hSbDE5YlBz eElQUi82aGhrVHFZQXRWSjk5VzQ5cGluRy9OL3I0c1J2aDkvUDVKbGx1N1JICkZHT3lWQmJqSlZS aTlFZUZqS2dKYjRMdnlMRVZEeGJoQUpHa1Y0OFBjeW5JNCtwRm5aUmFWSHM4ZzJiMXRmVmUKN3l2 d2F0ZkxyNXpESTVRYTZmWDZGbHRkWTlkb0NuODZmcXFZY1dZUVhOWVRYVmJBeHB0SFpXUTg0b3NO c1QxKwpWUUJVYnlMZVQ3ZFZwQVJGVllicmEzU2RXNndkZVpIcDNyeTNXMGhtVXBYSjlDZCtuUjg9 Cj1CTnloCi0tLS0tRU5EIFBHUCBTSUdOQVRVUkUtLS0tLQo= --===============5367386367043524075==--