public inbox for passt-dev@passt.top
 help / color / mirror / code / Atom feed
From: David Gibson <david@gibson.dropbear.id.au>
To: passt-dev@passt.top
Subject: [PATCH v3 00/11] Improve command dispatch in test scripts
Date: Mon, 12 Sep 2022 20:56:13 +1000	[thread overview]
Message-ID: <20220912105624.969382-1-david@gibson.dropbear.id.au> (raw)

[-- Attachment #1: Type: text/plain, Size: 3129 bytes --]

Sorry for the resend, but I found and fixed a couple of warts in this
series.

The test scripts need to dispatch commands in various different
contexts: on the host, in namespaces, and in guests.  Currently this
is done by running each context in a tmux pane and simulating typing
into that pane using tmux commands.  This has a number of problems:

 * It's slow
 * Getting the results from programs is tricky and error prone.  We
   can misparse output if:
     * The window isn't large enough
     * Commands produce output which looks like a shell prompt
     * The shell doesn't have the prompt we expect
 * A accidental keypress while the tests are running can mess up the
   test run
 * We have to issue explicit "echo $?" commands to check if things
   succeeded

This series adds a new subsystem to the test scripts for dispatching
commands which accurately preserves output (both stdout and stderr),
and exit codes.

Most of the testsuite is converted to this new system.  For now the
distro tests still rely on the old approach, since they have some
complications that require additional handling.  That does, alas, mean
we have some ugly transitional code remaining.  Oh well, we should be
able to clean that up at some point.

This series is pased on the previous series cleaning up the
performance tests.

Changes since v2:
 * Whitespace fixes based on review comments
 * Added proper copyright, license and banner information to nsholder
Changes since v1:
 * Removed a couple of commented debugging lines in patch 9/11
 * Removed a no longer necessary pipe-pane in patch 9/11

David Gibson (11):
  test: Correctly match "background" with "wait" commands
  test: Context execution helpers
  test: Allow a tmux pane to watch commands executed in contexts
  test: Integration of old-style pane execution and new context
    execution
  test: Issue host commands via context for most tests
  test: Use new-style contexts for passt pane in the pasta and passt
    tests
  test: Add nsholder utility
  test: Extend context system to run commands in namespace for pasta
    tests
  test: Use context system for guest commands
  test: Use context system for two_guests tests
  test: Use new-style command issue for passt_in_ns tests

 test/.gitignore           |   3 +
 test/Makefile             |  13 +-
 test/lib/context          | 125 +++++++++++++++++
 test/lib/layout           |  61 +++------
 test/lib/setup            | 274 +++++++++++++++++++-------------------
 test/lib/term             |  84 ++++++++++++
 test/lib/test             | 138 ++++++++-----------
 test/nsholder.c           | 139 +++++++++++++++++++
 test/passt.mbuto          |  32 ++++-
 test/run                  |   1 +
 test/shutdown/passt       |   4 +-
 test/shutdown/passt_in_ns |   4 +-
 test/tcp/passt            |   2 +
 test/tcp/passt_in_ns      |  17 ++-
 test/tcp/pasta            |   1 -
 test/udp/passt            |   1 +
 test/udp/passt_in_ns      |   5 +-
 test/udp/pasta            |   1 -
 18 files changed, 628 insertions(+), 277 deletions(-)
 create mode 100644 test/lib/context
 create mode 100644 test/nsholder.c

-- 
2.37.3


             reply	other threads:[~2022-09-12 10:56 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-09-12 10:56 David Gibson [this message]
2022-09-12 10:56 ` [PATCH v3 01/11] test: Correctly match "background" with "wait" commands David Gibson
2022-09-12 10:56 ` [PATCH v3 02/11] test: Context execution helpers David Gibson
2022-09-12 10:56 ` [PATCH v3 03/11] test: Allow a tmux pane to watch commands executed in contexts David Gibson
2022-09-12 10:56 ` [PATCH v3 04/11] test: Integration of old-style pane execution and new context execution David Gibson
2022-09-12 10:56 ` [PATCH v3 05/11] test: Issue host commands via context for most tests David Gibson
2022-09-12 10:56 ` [PATCH v3 06/11] test: Use new-style contexts for passt pane in the pasta and passt tests David Gibson
2022-09-12 10:56 ` [PATCH v3 07/11] test: Add nsholder utility David Gibson
2022-09-12 10:56 ` [PATCH v3 08/11] test: Extend context system to run commands in namespace for pasta tests David Gibson
2022-09-12 10:56 ` [PATCH v3 09/11] test: Use context system for guest commands David Gibson
2022-09-12 10:56 ` [PATCH v3 10/11] test: Use context system for two_guests tests David Gibson
2022-09-12 10:56 ` [PATCH v3 11/11] test: Use new-style command issue for passt_in_ns tests David Gibson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220912105624.969382-1-david@gibson.dropbear.id.au \
    --to=david@gibson.dropbear.id.au \
    --cc=passt-dev@passt.top \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://passt.top/passt

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for IMAP folder(s).