Re: [PATCH] test: Add Podman system test with bats for pasta

[Paul Holzinger <pholzing@redhat.com>]

On 23/08/2023 11:35, Stefano Brivio wrote:
> Ugly as hell, but we keep breaking things otherwise, and I keep
> forgetting to run this manually (as long as it's based on my local
> Podman setup, that's the only alternative).
>
> We need to clone the Podman repository as distribution packages don't
> contain test scripts, typically. While at it, build the latest
> version which is what really matters.
We actually ship a podman-tests rpm which contains `test/system` from 
the podman repo. Of course this will contain the test version based on 
the current shipped version and not the main branch.
But if you want to the with latest you have to clone from github of course.
>
> As we're planning anyway to revamp the test framework, I'd be
> inclined to just add this without too many thoughts, and have it as
> a nice-to-have requirement reminder for the new framework.
>
> Link: https://github.com/containers/podman/pull/19699
> Suggested-by: Paul Holzinger <pholzing@redhat.com>
> Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
> ---
>   test/README.md         |  4 ++--
>   test/pasta_podman/bats | 22 ++++++++++++++++++++++
>   test/run               |  4 ++++
>   3 files changed, 28 insertions(+), 2 deletions(-)
>   create mode 100644 test/pasta_podman/bats
>
> diff --git a/test/README.md b/test/README.md
> index 03c7f57..0936b04 100644
> --- a/test/README.md
> +++ b/test/README.md
> @@ -28,8 +28,8 @@ on a system, i.e. common utilities such as a shell are not included here.
>   
>   Example for Debian, and possibly most Debian-based distributions:
>   
> -    build-essential git jq strace iperf3 qemu-system-x86 tmux sipcalc bc
> -    clang-tidy cppcheck isc-dhcp-common psmisc linux-cpupower socat
> +    build-essential git jq strace iperf3 qemu-system-x86 tmux sipcalc bats bc
> +    catatonit clang-tidy cppcheck go isc-dhcp-common psmisc linux-cpupower socat
>       netcat-openbsd fakeroot lz4 lm-sensors qemu-system-arm qemu-system-ppc
>       qemu-system-misc qemu-system-x86 valgrind
>   
> diff --git a/test/pasta_podman/bats b/test/pasta_podman/bats
> new file mode 100644
> index 0000000..f36da7c
> --- /dev/null
> +++ b/test/pasta_podman/bats
> @@ -0,0 +1,22 @@
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +#
> +# PASST - Plug A Simple Socket Transport
> +#  for qemu/UNIX domain socket mode
> +#
> +# PASTA - Pack A Subtle Tap Abstraction
> +#  for network namespace/tap device mode
> +#
> +# test/pasta_podman/bats - Build Podman, run pasta system test with bats
> +#
> +# Copyright (c) 2022 Red Hat GmbH
> +# Author: Stefano Brivio <sbrivio@redhat.com>
> +
> +htools	git make go bats catatonit ip jq socat
> +
> +test	Podman system test with bats
> +
> +host	git -C __STATEDIR__ clone https://github.com/containers/podman.git
> +host	make -C __STATEDIR__/podman
> +hout	WD pwd
> +host	printf "[engine]\nhelper_binaries_dir=['__WD__']\n" > __STATEDIR__/containers.conf
> +host	PODMAN="__STATEDIR__/podman/bin/podman" CONTAINERS_CONF_OVERRIDE="__STATEDIR__/containers.conf" bats __STATEDIR__/podman/test/system/505-networking-pasta.bats
I recommend to use the `CONTAINERS_HELPER_BINARY_DIR` to set the 
directory over containers.conf, with your containers.conf change you 
will unset all other helper directories.
This may be fine for the pasta specific tests for now as we do not use 
other binaries such as netavark there but in the future it could change 
and leaves you wondering what broke.
> diff --git a/test/run b/test/run
> index 8f4f845..3b37663 100755
> --- a/test/run
> +++ b/test/run
> @@ -82,6 +82,10 @@ run() {
>   	test pasta_options/log_to_file
>   	teardown pasta_options
>   
> +	setup build
> +	test pasta_podman/bats
> +	teardown build
> +
>   	setup memory
>   	test memory/passt
>   	teardown memory