Contents: B C D E F J M O P R S T

Here is a glossary of terms used in this wiki:

Here is a short table that relates a few Jenkins terms to Fuego terms:

Jenkins term  ^	Fuego term  ^	Description  ^
slave	target or board	item being tested (Fuego defines a Jenkins slave for each board in the system, and calls it a target)
job	test	a collection of information needed to perform a single test
build	test run	the results from executing the test once

B [edit section]

base test script

this is a small script associated with a particular test. It runs on host (in the container) when a test is executed, and is turned into an expanded test script by the overlay generator.

benchmark

a type of test where one or more numeric metrics indicates the status of the test. See Benchmark parser notes for more information about processing these metrics.

board file

a file that describes (using environment variables) the attributes of a target board. This has the extension .board and is kept in the directory /userdata/conf/boards.

C [edit section]

console log

the full output of execution of a test from Jenkins. See Log files for details.

D [edit section]

devlog

the developer log for a test. See Log files for details.

Device

the name of a target board in the Fuego system.

device under test

in test terminology, this refers to the item being tested. In Fuego, this is also called the Device, or target.

E [edit section]

expanded test script

this is a script generated at test execution time by the overlay generator. It is generated when the test script (base test script) is executed, and environment variables are used to source various files and inputs for the final script.

F [edit section]

functional test

a type of test that returns a single pass/fail result, indicating whether the device under test. It may include lots of sub-tests.

J [edit section]

Jenkins

an advanced continuous integration system, used as the default front-end for the Fuego test framework. see Jenkins

log file

Several log files are created during execution of a test. For details about all the different log files, see Log files

job

In Jenkins termology, a job is a test

M [edit section]

metric

a numeric value measured by a benchmark test as the result of the test. This is compared against a threshold value to determine if the test passed or failed. See Benchmark parser notes

O [edit section]

ovgen.py

program to collect "overlay" data from various scripts and data files, and produce the final test script to run. see Overlay Generation

P [edit section]

parsed log

the test log file after it has been filtered by log_compare. See Log files for details.

parser.py

a python program, included with each Benchmark test, to scan the test log for benchmark metrics, check each against a reference threshold, and produce a plot.png file for the test. See parser.py and Benchmark parser notes for more information.

PLATFORM

defines the build platform for the device. This is used to select a set of environment variables to define and configure the toolchain for building programs for the intended test target.

R [edit section]

reference log

this file (called "reference.log") defines the regression threshhold (and operation) for each metric of a benchmark test. See reference.log and Benchmark parser notes

S [edit section]

syslog

the system log for a test. This is the system log collected during execution of a test. See Log files for details.

T [edit section]

test

This is a collection of scripts, jenkins configuration, source code, and data files used to validate some aspect of the device under test. See Test definition for more information.

test log

This is the log output from the actual test program on the target. There are multiple logs created during the execution of a test, and some might casually also be called "test logs". However, in this documentation, the term "test log" should be used only to refer to the test program output. See Log files for details.

test phases

Different phases of test execution defined by Fuego: pre_test, build, deploy, test_run, get_testlog, test_processing, post_test. For a description of phases see: Architecture#fuego_test_phases

test program

a program that runs on the target to execute a test and output the results. This can be a compiled program or a shell script (in which case the build step is empty)

test run

this is a single instance of a test execution, containing logs and other information about the run. This is referred to in Jenkins as a 'build'.

test script

the shell script that interfaces between the fuego core system and a test program. The script declares a tarfile, and functions to build, deploy and run the test. The test script is runs on the host. This is also called the 'base test script'. For details about the environment a script is run in or the functions it may call, see Variables, Core interfaces, and Test Script APIs

tools.sh

file containing the definition of toolchain variables for the different platforms installed in the container (and supported by the test environment) See tools.sh for details.