Summary

Fuego does not support arbitrary arrays of results data

Owner

Tim

Reporter

Daniel

Status

open

Priority

medium

Description [edit section]

Fuego currently does not support arrays of arbitrary size that can be populated dynamically. The reason is that reference.json is separate from parser.py, where the results data is separated.

Note: This issue, of dynamic lists of results, came up in Tim's test standards BOF session at Linaro Connect. It affects the ability to have static tguids for all tests, which is a core idea for comparing data between systems.

Basically, Fuego currently considers test set names, test case names and measure names to be statically defined. However, some tests run tests based on data determined at test run time.

This is a problem for how we currently declare reference values (measure thresholds) and units (currently found in criteria.json and reference.json respectively).

Notes [edit section]

Some ideas for how to solve this, without disturbing our current methods of encoding criteria and reference values, would be to: 1) allow wildcarding of

It might be possible to support a sparse set of tguids within a much larger space of possible tguids. For example, for cyclictest, we could support 1st thread latency, 2nd thread latency, etc. as static names, etc. without requiring that the 2nd one be present (it would depend on the spec and the parameters to the cyclictest program).

backlink

Fuego Issues List