From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=C6z7OtKm; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by passt.top (Postfix) with ESMTPS id 9F98B5A026F for ; Thu, 09 Oct 2025 21:29:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1760038147; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eB8mLhT3klK0RAwUAyaLoXF8CoNChHt6gnPDSKjiOes=; b=C6z7OtKmzlQULTQzxDXCxtqMw82D/IupFdKLDhH50/Vnoq3VUwpAVUQPM2JVn24HHuRmJO po5z3/sra0bya87icc0o0gWVyVF5eIAMtIxPYlsSCDDVFTU5ZUJj3GESe+Habnxrp4HHze MF/YO0oMuGbtaPTPIRV6U0suxYzubSg= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-632-jIo91zEENUm1HMyCvw-dng-1; Thu, 09 Oct 2025 15:29:06 -0400 X-MC-Unique: jIo91zEENUm1HMyCvw-dng-1 X-Mimecast-MFC-AGG-ID: jIo91zEENUm1HMyCvw-dng_1760038145 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-46e36686ca1so13123495e9.2 for ; Thu, 09 Oct 2025 12:29:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1760038144; x=1760642944; h=content-transfer-encoding:mime-version:organization:references :in-reply-to:message-id:subject:cc:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=eB8mLhT3klK0RAwUAyaLoXF8CoNChHt6gnPDSKjiOes=; b=gMuXRuco+eULb99vvEOo5vIrwM96YiVl63APps7AqCvGUfiT1uEo1TT0GygX2mrbaq Qod/M/t4S8gStOKQ/5UOuo+Xrkq2H8Xq9XxDx6F8NM0kkDPsKTXPptYv8kQGbTg5A6Pe yPnHmzT2vamMF6QIS4Jmrhd5ZmUBvrN/8M7eGdj1YNdbHe00/+lF9lo0UT7g+ySgoJsD usxonJOeKNfQOxRNQw6OK504f/zOSCRo3k3MFzisqX+NEaJHHtcpZmt1mEXmDC1XrmDB ySDXB34+w5et+jsbSdS/Gr/0ag2qDQ0NQqWfU4xM8nHZQtuXBzp5apOBORyOciFizw+Y xxAQ== X-Gm-Message-State: AOJu0Yxq1/iA1C5EFZbDtF4kJamvUsbFu2wH8ATKbRphG7nNtI12gRRh El1dfEz8PgaN7eKldk+AjOPQ3n27hZwmLXy/7f4qDUeaXK2TcwJX019AdfarGPilNvUpMmHgo+Z CAieGQW4B4GerTteI3oF643+R5kbYTtpdGB/fgA1Z2o9K7iYBEaoKJLTY4yiomg== X-Gm-Gg: ASbGncuEqVsPyq/dMhlGGrBHsp4YKWS2G9FoEJ3a6hgCKpoVcWa5inwZ6CZMgp5S5EF 7jxzyitOsbazaHMlYJTCBeALYtfxAR8c6ErDrgXZwrHYMvFAi6U7HUS1NFNpI1xWYBy0keqsM91 0lQ/zUQk2hvwKo2s9QJtY63K5rU5/DTeUXR03k3Au8nVCaFawzEZxqVzEEXSda8mZMns1MHZP2e Nu8ISgutlouihxCvac+WM9C1v8QLh7p404N1ToOiY25FhNFwcmidvwaDwuliFPJw5AfMfIhPkdq hCWmI7Dr0n1SPy0be6jFgIo095ks4D6JxONH77eVGaVBbGUomLVV6qYtc/DpBCQl2II/OAgNLA= = X-Received: by 2002:a05:6000:604:b0:424:2275:63b4 with SMTP id ffacd0b85a97d-4266e8e4cd7mr5931496f8f.61.1760038144319; Thu, 09 Oct 2025 12:29:04 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGMwgjoMW09QHueXwZMU4tWGVKnqleYa+H9VvZi9/qDTbwtctOl9VDJXt+wA9mHbbXyuRLJpw== X-Received: by 2002:a05:6000:604:b0:424:2275:63b4 with SMTP id ffacd0b85a97d-4266e8e4cd7mr5931483f8f.61.1760038143825; Thu, 09 Oct 2025 12:29:03 -0700 (PDT) Received: from maya.myfinge.rs (ifcgrfdd.trafficplex.cloud. [176.103.220.4]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-426ce5e0efasm446336f8f.41.2025.10.09.12.29.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Oct 2025 12:29:03 -0700 (PDT) Date: Thu, 9 Oct 2025 21:29:02 +0200 From: Stefano Brivio To: David Gibson Subject: Re: [PATCH 2/2] test: For missing static checkers, skip rather than failing tests Message-ID: <20251009212902.1b75082c@elisabeth> In-Reply-To: <20251009034358.1256908-3-david@gibson.dropbear.id.au> References: <20251009034358.1256908-1-david@gibson.dropbear.id.au> <20251009034358.1256908-3-david@gibson.dropbear.id.au> Organization: Red Hat X-Mailer: Claws Mail 4.2.0 (GTK 3.24.49; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: JWdTc0ouD721cJh_kj6HamE4Hoo5KYndeyIfSNn8TZs_1760038145 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Message-ID-Hash: 5MKMFCLCWNUE3G5U5Y7BYK6LIGF2IOSM X-Message-ID-Hash: 5MKMFCLCWNUE3G5U5Y7BYK6LIGF2IOSM X-MailFrom: sbrivio@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Thu, 9 Oct 2025 14:43:58 +1100 David Gibson wrote: > We run a bunch of static checkers as part of our testsuite. That's useful, > but it means that if a user doesn't have one of them installed, it fails > the entire testsuite. Alter our scripts to skip the test, rather than > failing outright if the checker tool is not installed. > > This requires exeter v0.4.4 or later. > > Signed-off-by: David Gibson > --- > test/build/static_checkers.sh | 17 +++++++++++++---- > test/lib/exeter | 10 +++++++++- > 2 files changed, 22 insertions(+), 5 deletions(-) > > diff --git a/test/build/static_checkers.sh b/test/build/static_checkers.sh > index 228b99ae..caccd183 100755 > --- a/test/build/static_checkers.sh > +++ b/test/build/static_checkers.sh > @@ -15,16 +15,25 @@ > > . $(dirname $0)/../exeter/sh/exeter.sh > > -exeter_register cppcheck make -C .. cppcheck > +do_check() { Nit: everywhere else in the test suite, we have kerneldoc-style comments (see lib/test). > + checker="$1" > + shift > + if ! which "${checker}"; then I think we should hide standard output here, just like we do for the *tools directives in test_one_line(), because if the check fails, it's a failure we already handled. > + exeter_skip "${checker} not available" > + fi > + make "${@}" "${checker}" > +} > + > +exeter_register cppcheck do_check cppcheck -C .. > exeter_set_description cppcheck "passt sources pass cppcheck" > > -exeter_register clang_tidy make -C .. clang-tidy > +exeter_register clang_tidy do_check clang-tody -C .. > exeter_set_description clang_tidy "passt sources pass clang-tidy" > > -exeter_register flake8 make flake8 > +exeter_register flake8 do_check flake8 > exeter_set_description flake8 "passt tests in Python pass flake8" > > -exeter_register mypy make mypy > +exeter_register mypy do_check mypy > exeter_set_description mypy "passt tests in Python pass mypy --strict" > > exeter_main "$@" > diff --git a/test/lib/exeter b/test/lib/exeter > index 530c6909..6ed92a16 100644 > --- a/test/lib/exeter > +++ b/test/lib/exeter > @@ -49,7 +49,15 @@ exeter() { > for __testid in $($EXETOOL list -- "$@"); do > __desc="$($EXETOOL desc -- "$@" -- "${__testid}")" > status_test_start "${__desc}" > - context_run host "$* '${__testid}'" && status_test_ok || status_test_fail > + status=0 > + context_run host "$* '${__testid}'" || status="$?" It would be nice to keep all variable names enclosed in curly brackets, for consistency, but also to avoid surprises. > + if [ "$status" = 0 ]; then > + status_test_ok > + elif [ "$status" = 77 ]; then I couldn't quite find out where 77 comes from. It's not specified in POSIX.1-2024 2.8.2 "Exit Status for Commands": https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V3_chap02.html#tag_19_08_02 and my version of which(1) just returns 1. Is this something from Bash? > + status_test_skip > + else > + status_test_fail > + fi > done > > cd .. -- Stefano