As an overview, the test.yaml file indicates where the source for the test comes from, it's license, the name of the test maintainer, a description of the test and tags for categorizing the test, and a formal list of parameters that are used by the test (what they mean and how to use them).
- indicates the version of package (in case of changes to the package schema). For now, this is always 1.
- has the full Fuego name of the test. Ex: Benchmark.iperf
- has an English description of the test
- has an SPDX identifier for the test. This is the main license of the test project that the Fuego test uses, if the project has a tarfile or git repo. Otherwise it reflects the license of any non-Fuego-specific materials in the test directory. In such case, the test directory should include a LICENSE file. Fuego materials (fuego_test.sh, spec.json, chart_config.json, etc.) are considered to be under the default Fuego license (which is BSD-3-Clause) unless otherwise specifically indicated in these files. The license identifier for this field should be obtained from https://spdx.org/licenses/
- the author or authors of the base test
- the maintainer of the Fuego materials for this test
- the version of the base test
- the version of Fuego materials for this test. This is a monotonically incrementing integer, starting at 1 for each new version of the base test.
- either Benchmark or Functional
- a list of tags used to categorize this test. This is intended to be used in an eventual online test store.
- a URL where the tarball was originally obtained from
- a git URL where the source may be obtained from
- a list of Debian package names that must be installed in the docker container in order for this test to work properly. This field is optional, and indicates packages needed that are beyond those included in the standard Fuego host distribution in the Fuego docker container.
- a list of parameters that may be used with this test, including their descriptions, whether they are optional or required, and an example value for each one
- a list of the files that are included in this test. This is used as the manifest for packaging the test (fuego_test.sh, and test.yaml are implicitly included in the packaging manifest).
The name of the parameter is the short name of the parameter, without the test name prefix (e.g. FUNCTIONAL_LTP). The parameter name is the key for the dictionary with that parameters attributes.
Each parameter has a dictionary with attributes describing it. The dictionary has the following fields (keys):
- 'description' - text description of the parameter
- 'example' - an example of the parameter
- 'optional' - indicates whether the test requires this parameter (test variable) to be set or not. The value of the 'optional' field must be one of 'yes' or 'no'.
The test variables may be described by the test.yaml file can be defined in one of multiple locations in the Fuego test system. Most commonly the test variables are defined in a spec for the test, but they can also be defined in the board file, or as a dynamic board variable.