Re: [PATCH/RFC] test: run static checkers with Avocado and JSON definitions

[David Gibson <david@gibson.dropbear.id.au>]

[-- Attachment #1: Type: text/plain, Size: 2146 bytes --]


I've now had a chance to play around with the patch, and I have some
further thoughts on that basis.

On Fri, Jul 05, 2024 at 09:35:28PM +1000, David Gibson wrote:

> On Sat, Jun 29, 2024 at 08:13:42AM -0400, Cleber Rosa wrote:
> Hi, sorry I've taken a while to reply to this.  I've been sick again,
> which has been no fun, plus I've been kind of preoccupied with getting
> the flow table ready.
> 
> > This adds a script and configuration to use the Avocado Testing
> > Framework to run, at this time, the static checkers.
> > 
> > The actual tests are defined using (JSON based) files, that are known
> > to Avocado as "recipes".  The JSON files are parsed and "resolved"
> > into tests by Avocado's "runnables-recipe" resolver.  The syntax
> > allows for any kind of test supported by Avocado to be defined there,
> > including a mix of different test types.
> 
> I'm not entirely clear.  Is this relying on the various extensions
> we've discussed and you've been working on?  Or is this using the
> older job API, but will become neater with the recent/coming
> extensions?

I see from experimenting that I can run the tests either with the
'run_avocado' script, or just by running
	avocado run test/avocado/static_checkers.json
or
	avocado run test/avocado

That's great! But.. without the config override in 'run_avocado' the
identifiers for each case aren't right: they just get called 'make' as
the name of the executable that's being run.  We really need a way to
set the identifier from the json file so that it's consistent.

The other thing I noticed is that it only works if you invoke either
avocado or run_avocado from the base of the passt tree.  I can see
why, given the implementation, but that's pretty surprising behaviour.

So, I guess we also need a way to specify the initial pwd in the json
- I guess it would need to be relative to where the json file itself
is to make sense.

-- 
David Gibson (he or they)	| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you, not the other way
				| around.
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]