From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=none (p=none dis=none) header.from=gibson.dropbear.id.au Authentication-Results: passt.top; dkim=pass (2048-bit key; secure) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.a=rsa-sha256 header.s=202508 header.b=FigueX2E; dkim-atps=neutral Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by passt.top (Postfix) with ESMTPS id 2717B5A0279 for ; Thu, 21 Aug 2025 04:48:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202508; t=1755744510; bh=9A43aDGy7QXBD0oTbS+Ves6eADiyeb6emTUJeC1fsIg=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=FigueX2Edo4tcRoTBZ/gC0Vqarak8I154O+9LGs0RohCmUQmTA9ItsDOrQX3d/k75 s3fxsTCECK0yS8kQQ1uBeGD6DQEUK1E7+KSGOLCZGKWCU8OudgRGtmKKnsWN7vWTat Xfxw5g5pPJSQHIdkAvLSjOpoU1dtsphurz0GEtgmp1IZTMqlLxp7ic/z2byvTeg655 xZ1DbIC/5NxwdL9NQ23xQ9tiRv560uWK97E1c0ty0Mu25EUWEjB6ZOi1MmIfA3jcrh HLR4UVYoikYN3yGGYYI3VXuzz6pe1TbBOSF3gFh8FI4dqVHgxNkRcVfyNEsUSEYk8U VAyskfHK1KRFg== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4c6nnV0vjRz4wd0; Thu, 21 Aug 2025 12:48:30 +1000 (AEST) Date: Thu, 21 Aug 2025 12:48:27 +1000 From: David Gibson To: Stefano Brivio Subject: Re: [PATCH v5 0/4] RFC: New proof-of-concept based exeter tests Message-ID: References: <20250820105456.1281906-1-david@gibson.dropbear.id.au> <20250820224048.413d9f2a@elisabeth> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="g53S2DAjz77Nh3e3" Content-Disposition: inline In-Reply-To: <20250820224048.413d9f2a@elisabeth> Message-ID-Hash: E2QE7M6C34WQL7S25WGSMJJG6HCERCAJ X-Message-ID-Hash: E2QE7M6C34WQL7S25WGSMJJG6HCERCAJ X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --g53S2DAjz77Nh3e3 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 20, 2025 at 10:40:48PM +0200, Stefano Brivio wrote: > On Wed, 20 Aug 2025 20:54:52 +1000 > David Gibson wrote: >=20 > > Here's a new approach to building passt tests with exeter. This new > > one no longer uses Avocado in the default case, although it would > > still be possible to manually run the exeter based tests with Avocado. > >=20 > > Here's another draft of my work on testing passt with Avocado and the > > exeter library I recently created. It includes Cleber's patch adding > > some basic Avocado tests and builds on that. > >=20 > > For now this only does simple tests, to show how the integration could > > work. It adds some new trivial "smoke tests" and converts the linter > > and build checks to exeter. More complex tests will require building > > the sinte/pesto library we've discussed. A lot of the work for that > > already exists in my earlier exeter test series, but it will need some > > rework to split it into a separate component. > >=20 > > v5: > > * Updated according to Stefano's review > > - Fixed a number of whitespace errors > > - Improved many comments and variable names to make things clearer > > * New patch adding parallel test execution with BATS > > * Improved autodetection of exeter tests using "exetool probe" >=20 > This works on my setup and looks good to me, I just have two comments: >=20 > - test names are still the same as before (not exactly descriptive, say, > 'make_passt'). I already reported this on v4, I'm not sure what was > your conclusion about it Sorry, I missed that comment on v4. exeter test ids are by design machine-friendly identifiers more than they are human-friendly names or descriptions. There are a few reasons for that: * The ids need to be passed around between test and runner both on the command line and via stdio. Limiting them to characters that are identifier friendly in most languages significantly reduces the chances of screwing up quoting. * In some existing Python cases, and maybe more language cases in future, the ids are auto-generated, e.g. for a matrix or composition of tests. That works more naturally for identifiers than names/descriptions. * Identifiers are more amenable to structured formatting grouping related tests together, which is useful for filtering out groups of test by glob/regexp. * I like having a succinct id to refer to tests by rather than a waffly English description I'm not opposed to having an (optional) human-readable name or description for tests in addition to the id. It would complexify the exeter protocol, of course, which I'm trying to keep super simple. Then again, I have several other things in mind that would need per-test metadata, so it's probably is worth it. > - I didn't check (yet) what happens when I run this as ./ci (for > example, from the pre-push hook), if generated web links are still > okay. I'll do that soon unless you can have a look first I don't really know how to check that. I don't think there's any reason it wouldn't work. --=20 David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson --g53S2DAjz77Nh3e3 Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEO+dNsU4E3yXUXRK2zQJF27ox2GcFAmimiPoACgkQzQJF27ox 2Gdi0A/8CTNc2X7+9sN72o6Zat/KDPwZvxt7BSEjE0QFsmC+4ixI1rHnTapz+pgw OtV8KRVspmYAHCEVutT5HBRQUnhLICwXpxCSCwkE6HzOVCfH//f2lvWSWGQq/U48 J9GopBF2vCkEGxISflY0T/c6ei5Ne7lmP48cldOJCMDcO34oQmqG7s+2Q2p+BnX6 rRuXO03urYxxUd3lx8qQ9uvySW76aiDKaf7aFtYW8YvL4n7xKNaOdt7lE8Q0LbJ5 GS9eZ5W9Cv6lKcNabIORlHfCHACnCINWphBgyGTlrN43LREVs++F51E6NRO+buVN tBDb/i5OjeCwPE17BcBWvuzHsGEfRSkxLHQgimoGuA/M1UpiG7goKbFkCvNvQHPt lwoiHGZb8OsPbfwpH7maufxYCNBZUES6nIYvEcJ85+8TYk+mRB4WZboadjtjFWjb nsG0T1+p4Vx1YcyxqULBvcS4MIsv62pz3NwJahM3piJh3zpgBFbodOVH49NQ7oES kT+J9QS/jTXweFaI4m24SFnuv5OczOkbLWRwaCcyvZfE0uY1Fk/zWG3QK/CJR4ZY PlOBojcpoO8R5U10luVJ2GCI2XOD+p6i+qyYaiDhjKu4H29LElCRk9zUIiw52i0J u14ODks57tMAGz2xtqQridoktPEGdUMmyjcX5c/vii8k6EzRVjU= =URFf -----END PGP SIGNATURE----- --g53S2DAjz77Nh3e3--