{{TableOfContents}}

This page has information about the project to de-couple Fuego's
test log parser, to make it a standalone program.

Neil Williams had some good feedback (some of it negative) on the
automated testing mailing list.

This page seeks to organize that content, and capture ideas about
the process of making Fuego's parser standalone.

= vision =
Create a standalone test program output parser, that can be used
by multiple test systems, so that parsing is not baked into
each test system.

Let it develop on its own, independent of a particular test system.

= input =
Run artifacts from a test program
 * stdout
 * kernel output (console log during run)
 * stderr
 * return code
 * trace during run
 * system log during run (with items from the test program)

There can be multiple channels of information to extract data from
during a test program run.

== format, markers or tags ==
 * TAP
 * LAVA

= output =
Standardized output in multiple formats:
 * junit
 * xunit
 * kernelci format
 * squad format
 * run.json (should be similar to kernelci)


= current dependencies on Fuego =
 * Fuego environment variables
   * FIXTHIS - list vars here
 * Fuego paths?

= Miscelaneous notes =
== Tim's notes ==
Up to Fuego 1.5 we invoke the parser with:
{{{#!YellowBox
   run_python $PYTHON_ARGS $FUEGO_CORE/engine/tests/${TESTDIR}/parser.py
}}}

(with a whole lot of Fuego-specific environment variables).

I think it would be good to refactor this, so that the fuego core (functions.sh)
calls a single program, indicating the test, the log file, and a parser name.
Many fuego parsers could be combined by declaring a single regex pattern
in the fuego_test.sh (similar to what is done with log_compare).

So, something like the following instead:
{{{#!YellowBox
   run_python $PYTHON_ARGS --log=$FUEGO_RW/logs/.../testlog.txt --test=$TESTDIR --parser=TAP13
}}}
or
{{{#!YellowBox
   run_python $PYTHON_ARGS --log=$FUEGO_RW/logs/.../testlog.txt --test=$TESTDIR \
     --parser=regex-2groups --parser-arg="regex_string= ^TEST-(\d+) (.*)$"
}}}

== Daniel's notes ==
I was thinking about a new python library called (e.g.: "fuegoparse") that you can use from
any project (e.g.: install it with pip3) and also be called from the command line.

Example usage as a library:

{{{#!YellowBox
import fuegoparse
import json

testparser = fuegoparse.TestlogParser(test="LTP", output_format="fuego_run_json")
data = testparser.parse("./testlog.txt")
with open('run.json', 'w') as outfile:
    json.dump(data, outfile)
}}}

Example as a command:
{{{#!YellowBox
  $ fuegoparse -t LTP --log ./testlog.txt --format fuego_run_json -o run.json
}}}

Other output formats could be TAP, KernelCI, Squad, Junit, etc.

This could become a whole new project (the first spinoff of the Fuego project!).

We also need to think about isolating other parts (cross-build, dependency checks, transport abstractions), 
so they can be reusable. And finally be able to say that Fuego is just glue code
that puts all those parts together.