From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by passt.top (Postfix) with ESMTPS id 7E60B5A004F for ; Mon, 15 Jul 2024 06:15:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202312; t=1721016929; bh=KBhs7gQTVSg17SUFzfY/+IrZn5tY5ahCHfusd9FqH+Q=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=nOlaEtboIN9YZ0J6P1SqRREfVSRd+YOCB05X3H3/4ZNAPNBK6Eoxk+jBFpApKPgQG 9/aFCvkxNYeiXx/GWy40xfzMmHX1E6CBp4QxphAyh5k7RAs+4+uFmnoVEidZKoIpvS +56P63dIiNisIi+hxkeBvTTN4t1COpwQiEhGl5X/H5GgC46C0J4rRGCIGPLulNQilK keVqllIpYxvbhcQQ1LeoRpLMS5aFasTltr5zi5ULrJQJuE9EjwwVWQTq+nGaALuIYG RGK4b48kkKvqfmpZmKdzcmHGQjo8p709JTVD7Dfhw71yUh1d/e6r7pMJEzAcdquLEv 6SJxD3PerQeYQ== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4WMplP2tWvz4x04; Mon, 15 Jul 2024 14:15:29 +1000 (AEST) Date: Mon, 15 Jul 2024 14:15:14 +1000 From: David Gibson To: Cleber Rosa Subject: Re: [PATCH/RFC] test: run static checkers with Avocado and JSON definitions Message-ID: References: <20240629121342.3284907-1-crosa@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="SJs/4unF6sx9r+8H" Content-Disposition: inline In-Reply-To: Message-ID-Hash: T7DWEZGZ356JLXIEFJBTQFDYY6RMGLSE X-Message-ID-Hash: T7DWEZGZ356JLXIEFJBTQFDYY6RMGLSE X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top, David Gibson X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --SJs/4unF6sx9r+8H Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I've now had a chance to play around with the patch, and I have some further thoughts on that basis. On Fri, Jul 05, 2024 at 09:35:28PM +1000, David Gibson wrote: > On Sat, Jun 29, 2024 at 08:13:42AM -0400, Cleber Rosa wrote: > Hi, sorry I've taken a while to reply to this. I've been sick again, > which has been no fun, plus I've been kind of preoccupied with getting > the flow table ready. >=20 > > This adds a script and configuration to use the Avocado Testing > > Framework to run, at this time, the static checkers. > >=20 > > The actual tests are defined using (JSON based) files, that are known > > to Avocado as "recipes". The JSON files are parsed and "resolved" > > into tests by Avocado's "runnables-recipe" resolver. The syntax > > allows for any kind of test supported by Avocado to be defined there, > > including a mix of different test types. >=20 > I'm not entirely clear. Is this relying on the various extensions > we've discussed and you've been working on? Or is this using the > older job API, but will become neater with the recent/coming > extensions? I see from experimenting that I can run the tests either with the 'run_avocado' script, or just by running avocado run test/avocado/static_checkers.json or avocado run test/avocado That's great! But.. without the config override in 'run_avocado' the identifiers for each case aren't right: they just get called 'make' as the name of the executable that's being run. We really need a way to set the identifier from the json file so that it's consistent. The other thing I noticed is that it only works if you invoke either avocado or run_avocado from the base of the passt tree. I can see why, given the implementation, but that's pretty surprising behaviour. So, I guess we also need a way to specify the initial pwd in the json - I guess it would need to be relative to where the json file itself is to make sense. --=20 David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson --SJs/4unF6sx9r+8H Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEO+dNsU4E3yXUXRK2zQJF27ox2GcFAmaUolEACgkQzQJF27ox 2Gejrg//WoUBPIB+0RaoEPPWAt79/g7U+/V0S+5S0lLwJ9i/XXjxEy+MHw4ytILP m3y9gupdy0XieCVl2Vjgt2M1ZxOco+hFXH7rvztlNuS8qV8xQLN5IFtr+GDMQ6AN rS4/7aExBnw06l0TRdDedCEq7bOFXGhvidzp8BxO4bunwaTe7EosX4zDaroH22N8 FL9NfKHzIY6HS/DIzwzbJE5P5UrdyEmkx+PZtgLqZj5TSSiLZxjlhi43xZzX7jSi yBfGSPBx1bJKQXHptLfQw+nIP/fJO5J0YNH7C1xUXIm9Uhr1pLy809YgNYeeEXoH ddRXudHVdbwlZO4+TJdlG6xcRU1GysnWll1xHKFETBK4rw80BaLQR7x6QNfq9nk5 wn/G7kCJYxhCfCsibhR7nZmwEfieRin6V9LwSc/ysS7K45R3lZKc8zVlc3jUrO98 Excka+5yqgtbVYngJdGXCSsdk2kwa9IkVlpICJ84Gc5fP+hJ+6oiwzOBD/3Igy84 akOBVDxLANDCbSx6ylejGMMRk4jzfmqNlpmKfCu/oRjqKmXdLXnU7U9uxdxVvSl3 e+5YvlINfHatd/GlRbV3w6FaW/5OjkslmU5bL/Qvy+W01PN3qn95/Iv9pMBnsqL+ nFNr3yRtVfZ2S/2UI65ZjcQAFdCWBb048asbiLEz0cnrKAZJhZE= =hCcC -----END PGP SIGNATURE----- --SJs/4unF6sx9r+8H--