Fuego tests produce a single, unified, output format.
|
Fuego tests produce a single, unified, output format.
|
The test results are saved in a file called run.json,
and the fields of the format are described on that page.
|
The test results are saved in a file called [[run.json]],
and the fields of the format are described on that page.
|
The reason for having a unified format is to allow consistent
processing of results from any test. For example, with a standard
output format it is possible to write visualization tools and
report generators that operate on the results from any test.
It is possible to transport test results between systems, and to
convert results to other standard formats (such as Xunit, which
is supported by a number of external frameworks and test systems).
|
The reason for having a unified format is to allow consistent
processing of results from any test. For example, with a standard
output format it is possible to write visualization tools and
report generators that operate on the results from any test.
It is possible to transport test results between systems, and to
convert results to other standard formats (such as Xunit, which
is supported by a number of external frameworks and test systems).
|
The parser for each test is responsible for extracting test results
and presenting them in a python data structure to the Fuego
results handling library. This library then evaluates the results
for final test status determination, and stores the results in
the log directory for that run of the test.
|
The parser for each test is responsible for extracting test results
and presenting them in a python data structure to the Fuego
results handling library. This library then evaluates the results
for final test status determination, and stores the results in
the log directory for that run of the test.
|
Additional features of Fuego will be built on this unified format
in the future.
|
Additional features of Fuego will be built on this unified format
in the future.
|
|
= Known deficiencies in Fuego's format =
The format was introduced in Fuego version 1.2, and is still somewhat
a work in progress. Some additional features are considered for inclusion
in the format in future releases.
|
Fuego parsers do not (as of version 1.2.1) put error message information into the format - nor does the current 1.0 schema specify fields for such information.
|
Fuego parsers do not (as of version 1.2.1) put error message information into the format - nor does the current 1.0 schema specify fields for such information.
|
Additional meta-data about tests is currently stored in additional JSON
files criteria.json and reference.json.
|
Additional meta-data about tests is currently stored in additional JSON
files [[criteria.json]] and [[reference.json]].
|