Re: [PATCH] test: Add rudimentary support to run selected tests only

[David Gibson <david@gibson.dropbear.id.au>]

[-- Attachment #1: Type: text/plain, Size: 4395 bytes --]

On Fri, Oct 07, 2022 at 02:47:32AM +0200, Stefano Brivio wrote:
> To keep this simple, only support tests that have corresponding setup
> and teardown functions implied by their path. For example:
> 
>   ./run passt/ndp
> 
> will trigger the 'passt' setup and teardown functions.
> 
> This is not really elegant, but it looks robust, and while David is
> considering proper alternatives, it should be quite useful.
> 
> Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com>

I'm not sure I'd go so far as to call it "robust".  AFAICT it will
break for cases where the setup function isn't named after the test
dir (e.g. for perf/* which need the passt_in_ns setup).  And, of
course, it's still the user's responsibility to handle inter-test
dependencies (dhcp needs ndp to run first , tcp & udp need dhcp to run
first , etc.)

But still, it's pretty simple and looks useful despite those
limitations so,

Reviewed-by: David Gibson <david(a)gibson.dropbear.id.au>


> ---
>  test/README.md | 16 ++++++++++++++++
>  test/lib/term  |  2 +-
>  test/run       | 37 ++++++++++++++++++++++++++++++++++++-
>  3 files changed, 53 insertions(+), 2 deletions(-)
> 
> diff --git a/test/README.md b/test/README.md
> index 72a4986..09ad05d 100644
> --- a/test/README.md
> +++ b/test/README.md
> @@ -85,6 +85,22 @@ variable settings: DEBUG=1 enables debugging messages, TRACE=1 enables tracing
>  
>      PCAP=1 TRACE=1 ./run
>  
> +## Running selected tests
> +
> +Rudimentary support to run a list of selected tests, without support for
> +dependencies, is available. Tests need to have a setup function corresponding to
> +their path. For example:
> +
> +    ./run passt/ndp passt/dhcp pasta/ndp
> +
> +will call the 'passt' setup function (from lib/setup), run the two corresponding
> +tests, call the 'passt' teardown function, the 'pasta' setup, run the pasta/ndp
> +test, and finally tear down the 'pasta' setup.
> +
> +Note that requirements on steps implemented by related tests are not handled.
> +For example, if the 'passt/tcp' needs guest connectivity set up by the
> +'passt/ndp' and 'passt/dhcp' tests, those need to be listed explicitly.
> +
>  ## Continuous integration
>  
>  Issuing:
> diff --git a/test/lib/term b/test/lib/term
> index eade2cd..1b42df8 100755
> --- a/test/lib/term
> +++ b/test/lib/term
> @@ -650,7 +650,7 @@ run_term() {
>  		asciinema rec --overwrite "${STATEBASE}/demo.uncut" -c "$TMUX /bin/sh -c './run_demo from_term'"
>  		video_postprocess "${STATEBASE}/demo.uncut"
>  	else
> -		$TMUX /bin/sh -c './run from_term'
> +		$TMUX /bin/sh -c "./run from_term ${*}"
>  	fi
>  }
>  
> diff --git a/test/run b/test/run
> index 4bb9cd8..1ae270e 100755
> --- a/test/run
> +++ b/test/run
> @@ -127,6 +127,37 @@ run() {
>  	return 0
>  }
>  
> +# run_selected() - Run list of tests, with setup/teardown based on test path
> +# $@:	List of tests
> +run_selected() {
> +	mkfifo $STATEBASE/log_pipe
> +
> +	term
> +	VALGRIND=1
> +
> +	__setup=
> +	for __test; do
> +		if [ "${__test%%/*}" != "${__setup}" ]; then
> +			[ -n "${__setup}" ] && teardown "${__setup}"
> +			__setup="${__test%%/*}"
> +			setup "${__setup}"
> +		fi
> +
> +		test "${__test}"
> +	done
> +	teardown "${__setup}"
> +
> +	log "PASS: ${STATUS_PASS}, FAIL: ${STATUS_FAIL}"
> +
> +	pause_continue \
> +		"Press any key to keep test session open"	\
> +		"Closing in "					\
> +		"Interrupted, press any key to quit"		\
> +		9
> +
> +	return 0
> +}
> +
>  # demo() - Simpler path for demo purposes
>  demo() {
>  	mkfifo $STATEBASE/log_pipe
> @@ -160,11 +191,15 @@ demo() {
>  [ "$(basename "${0}")" = "run_demo" ] && DEMO=1
>  
>  if [ "${1}" = "from_term" ]; then
> +	shift
> +
>  	exec > ${LOGDIR}/script.log 2>&1
>  	[ ${DEBUG} -eq 1 ] && set -x
>  	cd ..
>  	if [ ${DEMO} -eq 1 ]; then
>  		demo
> +	elif [ -n "${1}" ]; then
> +		run_selected ${*}
>  	else
>  		run
>  	fi
> @@ -176,7 +211,7 @@ else
>  	:> "${LOGFILE}"
>  	STATEBASE="$(mktemp -d --tmpdir passt-tests-XXXXXX)"
>  	trap "cleanup" EXIT
> -	run_term
> +	run_term ${*}
>  fi
>  
>  [ ${DEMO} -eq 1 ] && exit 0

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]