Fuego naming rules
Here are some rules and conventions for naming things in Fuego:
- a Fuego test name must have one of the following prefixes:
- the name following the prefix is known as the base test name, and has the following rules:
- it may only use letters, numbers and underscores
- that is - no dashes
- it may use upper and lower case letters
- it may only use letters, numbers and underscores
- All test definition materials reside in a directory with the full test name:
- e.g. Functional.hello_world
- the base script file has the name fuego_test.sh
- the default spec file for a test has name spec.json
- the default criteria file for a test has the name criteria.json
- the reference file for a test has the name reference.json
- the parser module for a test has the filename parser.py
Every test should have at least one spec, called the 'default' spec. If no spec file exists for a test, then Fuego generates a 'default' spec (on-the-fly), which is empty (ie. has no test variables).
- boards are defined by files in the /fuego-ro/boards directory
- their filenames consists of the board name, with the suffix ".board"
- e.g. beaglebone.board
- a board name should have only letters, numbers and underscores.
- specifically, no dashes, periods, or other punctuation is allowed
- A Jenkins node corresponding to a board must have the same name as the board.
- e.g. beaglebone
- A Jenkins job is used to execute a test.
- Jenkins job names should consist of these parts: <board>.<spec>.<test_name>
- e.g. beaglebone.default.Functional.hello_world
The parts of a run id are separated by dashes, except that the separator between the host and the board is a colon.
A fully qualified (global) run identifier consist of the following parts:
- test name
- spec name
- build number
- the word "on"
FIXTHIS - global run ids should include timestamps to make them globally unique for all time
A shortened run identifier may omit the "on" and "host". This is referred to as a local run id, and is only valid on the host where the run was produced.
- A Fuego timestamp has the format: YYYY-mm-dd_HH:MM:SS
- e.g. 2017-03-29_10:25:14
- times are expressed in localtime (relative to the host where they are created)
They consist of a several parts, some of which may be omitted in some circumstances
The parts are:
- testsuite name
- testset name
- testcase name
- measure name
Legal characters for these parts are letters, numbers, and underscores. Only testset names may include a period ("."), as that is used as the separator between constituent parts of the identifier.
testcase identifiers should be consistent from run-to-run of a test, and should be globally unique.
test identifiers may be in fully-qualified form, or in shortened form - missing some parts. The following rules are used to convert between from shortened forms to fully-qualified forms.
If the testsuite name is missing, then the base name of the test is used.
- e.g. Functional.jpeg has a default testsuite name of "jpeg"
If the testset name is missing, then the testset name is "default".
A test id may refer to one of two different items:
- a testcase id
- a measure id
A fully qualified test identifier consists of a testsuite name, testset name and a testcase name. Shortened names may be used, in which case default values will be used for some parts, as follows:
If a result id has only 1 part, it is the testcase name. The testset name is considered to be "default", and the testsuite name is the base name of the test.
That is, for the fuego test Functional.jpeg, a shortened tguid of "test4", the fully qualified name would be:
If a result id has 2 parts, then the first part is the testset name and the second is the testcase name, and the testsuite name is the base name of the test.
A shortened measure id may not consist of less than 2 parts. If it only has 2 parts, the first part is the testcase id, and the second part is the measure name. In all cases the last part of the name is the measure name, the second-to-last part of the name is the testcase name.
If there are three parts, the first part is the testset name.
Some test variable prefixes and suffixes are used in a consistent way.
- PROGRAM_FOO - require program 'foo' on target. The program name is upper-cased, punctuation or spaces are replaced with '_', and the name is prefixed with 'PROGRAM_'. The value of variable is full path on target where program resides.
- ex: PROGRAM_BC=/usr/bin/bc
- HEADER_FOO - require header file 'foo' in SDK. The header filename is stripped of its suffix (I donít know if that's a good idea or not), upper-cased, punctuation or spaces are replaced with '_', and the name is prefixed with 'HEADER_'. The value of variable is the full path in the SDK of the header file:
- ex: HEADER_FOO=/opt/poky2.1.2/sysroots/x86_64-pokysdk-linux/usr/include/foo.h
- SDK_LIB_FOO - require 'foo' library in SDK. The library filename is stripped of the 'lib' prefix and .so suffix, upper-cased, punctuation and spaces are replaced with '_', and the name is prefixed with 'SDK_LIB_'. The value of the variable is the full path in the SDK of the library.
- ex: SDK_LIB_FOO=/opt/poky2.1.2/sysroots/x86_64-pokysdk-linux/usr/lib/libfoo.so
- Note that in case a static library is required (.a), then the variable name should include that suffix:
- ex: SDK_LIB_FOO_A=/opt/poky1.2.1/sysroots/x86_64-pokysdk-linux/usr/lib/libfoo.a
- TARGET_LIB_FOO - require 'foo' library on target. The library filename is stripped
of the 'lib' prefix and .so suffix (not sure this is a good idea, as we potentially lose a library
version requirement), upper-cased, punctuation and spaces are replaced with '_', and the name is prefixed with 'TARGET_LIB_'. The value of the variable is the full path of the library on the target board.
- ex: TARGET_LIB_FOO=/usr/lib/libfoo.so