Here is a glossary of terms used in this wiki:
|
{{TableOfContents}}
Here is a glossary of terms used in this wiki:
|
Here is a short table that relates a few Jenkins terms to Fuego terms:
{{{#!Table
show_sort_link=0
show_edit_links=0
||Jenkins term||Fuego term||Description||
||slave||none||this is a long-running jenkins process, that executes jobs. It is usually (?) assigned to a particular node||
||node||board||item being tested (Fuego defines a Jenkins node for each board in the system)||
||job||test||a collection of information needed to perform a single test||
||none||request||a request to run a particular test on a board||
||build||run (or 'test run')||the results from executing the job or test||
||none||plan||the plan has the list of tests and how to run them (which variation, or 'spec' to use)||
||none||spec||the spec indicates a particular variation of a test||
}}}
|
Here is a short table that relates a few Jenkins terms to Fuego terms:
{{{#!Table
show_sort_link=0
show_edit_links=0
||Jenkins term||Fuego term||Description||
||slave||''none''||this is a long-running jenkins process, that executes jobs. It is usually (?) assigned to a particular node||
||node||board||item being tested (Fuego defines a Jenkins node for each board in the system)||
||job||test||a collection of information needed to perform a single test||
||''none''||request||a request to run a particular test on a board||
||build||run (or 'test run')||the results from executing the job or test||
||''none''||plan||the plan has the list of tests and how to run them (which variation, or 'spec' to use)||
||''none''||spec||the spec indicates a particular variation of a test||
}}}
|
Benchmark parser notes for more information about processing these metrics.
; binary package: a tarfile containing the materials that would normally be deployed to the board for execution.
; board file: a file that describes (using environment variables) the attributes of a target board. This has the extension .board and is kept in the directory /fuego-ro/boards.
|
= B =
; base test script: this is a small script associated with a particular test. It provides a set of test functions that are executed on the host (in the container) when a test is run.
; benchmark: a type of test where one or more numeric metrics indicates the status of the test. See [[Benchmark parser notes]] for more information about processing these metrics.
; binary package: a tarfile containing the materials that would normally be deployed to the board for execution.
; board file: a file that describes (using environment variables) the attributes of a target board. This has the extension .board and is kept in the directory /fuego-ro/boards.
|
Log files for details.
|
= C =
; console log: the full output of execution of a test from Jenkins. See [[Log files]] for details.
|
Log files for details.
; Device: the name of a target board in the Fuego system.
; device under test: in test terminology, this refers to the item being tested. In Fuego, this may also be called the Device, target, or node.
; distribution: This refers to a set of software that is running on a Linux machine. Example "distributions" are Debian, Angstrom or Android. The distribution defines file locations, libraries, utilities and several important facilities and services of the machine (such as the init process, and the logger).
|
= D =
; devlog: the developer log for a test. See [[Log files]] for details.
; Device: the name of a target board in the Fuego system.
; device under test: in test terminology, this refers to the item being tested. In Fuego, this may also be called the ''Device'', ''target'', or ''node''.
; distribution: This refers to a set of software that is running on a Linux machine. Example "distributions" are Debian, Angstrom or Android. The distribution defines file locations, libraries, utilities and several important facilities and services of the machine (such as the init process, and the logger).
|
|
= F =
; functional test: a type of test that returns a single pass/fail result, indicating whether the device under test. It may include lots of sub-tests.
|
Jenkins
; log file: Several log files are created during execution of a test. For details about all the different log files, see Log files
; job: In Jenkins terminology, a job is a test
|
= J =
; Jenkins: an advanced continuous integration system, used as the default front-end for the Fuego test framework. see [[Jenkins]]
; log file: Several log files are created during execution of a test. For details about all the different log files, see [[Log files]]
; job: In Jenkins terminology, a job is a test
|
Benchmark parser notes
= O =
; ovgen.py: program to collect "overlay" data from various scripts and data files, and produce the final test script to run. see Overlay Generation
|
= M =
; metric: a numeric value measured by a benchmark test as the result of the test. This is compared against a threshold value to determine if the test passed or failed. See [[Benchmark parser notes]]
= O =
; ovgen.py: program to collect "overlay" data from various scripts and data files, and produce the final test script to run. see [[Overlay Generation]]
|
Log files for details.
; parser.py: a python program, included with each Benchmark test, to scan the test log for benchmark metrics, check each against a reference threshold, and produce a plot.png file for the test. See parser.py and Benchmark parser notes for more information.
|
= P =
; parsed log: the test log file after it has been filtered by log_compare. See [[Log files]] for details.
; parser.py: a python program, included with each Benchmark test, to scan the test log for benchmark metrics, check each against a reference threshold, and produce a plot.png file for the test. See [[parser.py]] and [[Benchmark parser notes]] for more information.
|
- provision
- To provision a board is to install the system software on it. Some board control systems re-provision a board for every test. In general, Fuego runs a series of tests with a single system software installation.
|
; provision: To provision a board is to install the system software on it. Some board control systems re-provision a board for every test. In general, Fuego runs a series of tests with a single system software installation.
|
reference.log and Benchmark parser notes
|
= R =
; reference log: this file (called "reference.log") defines the regression threshhold (and operation) for each metric of a benchmark test. See [[reference.log]] and [[Benchmark parser notes]]
|
Test_variables#Spec_variables
; stored variable: a test variable that is stored in a read/write file, and can be updated manually or programmatically. See Test_variables#Stored_variables
; syslog: the system log for a test. This is the system log collected during execution of a test. See Log files for details.
|
= S =
; spec variable: a test variable that comes from a spec file. See [[Test_variables#Spec_variables]]
; stored variable: a test variable that is stored in a read/write file, and can be updated manually or programmatically. See [[Test_variables#Stored_variables]]
; syslog: the system log for a test. This is the system log collected during execution of a test. See [[Log files]] for details.
|
Fuego Object Details for more information.
; test log: This is the log output from the actual test program on the target. There are multiple logs created during the execution of a test, and some might casually also be called "test logs". However, in this documentation, the term "test log" should be used only to refer to the test program output. See Log files for details.
; test package: This is a packaged version of a test, including all the materials needed to execute the test on another host. See Test package system
; test phases: Different phases of test execution defined by Fuego: pre_test, build, deploy, test_run, get_testlog, test_processing, post_test. For a description of phases see: Architecture#fuego_test_phases
; test program: a program that runs on the target to execute a test and output the results. This can be a compiled program or a shell script (in which case the build step is empty)
; test run: this is a single instance of a test execution, containing logs and other information about the run. This is referred to in Jenkins as a 'build'.
; test script: the shell script that interfaces between the fuego core system and a test program. The script declares a tarfile, and functions to build, deploy and run the test. The test script runs on the host. This is also called the 'base test script'. For details about the environment that a script runs in or the functions it may call, see Variables, Core interfaces, and Test Script APIs
; test variable: This is the name of a variable available to the a test during it's execution. See Test variables
; test variable script: this is a script generated at test execution time by the overlay generator. It is generated when the test script (base test script) is executed, and environment variables are used to source various files and inputs for the final script. It is called 'prolog.sh'
; TOOLCHAIN: defines the toolchain or SDK for the device. This is used to select a set of environment variables to define and configure the toolchain for building programs for the intended test target.
; tools.sh: file containing the definition of toolchain variables for the different platforms installed in the container (and supported by the test environment) See tools.sh for details.
|
= T =
; test: This is a collection of scripts, jenkins configuration, source code, and data files used to validate some aspect of the device under test. See [[Fuego Object Details]] for more information.
; test log: This is the log output from the actual test program on the target. There are multiple logs created during the execution of a test, and some might casually also be called "test logs". However, in this documentation, the term "test log" should be used only to refer to the test program output. See [[Log files]] for details.
; test package: This is a packaged version of a test, including all the materials needed to execute the test on another host. See [[Test package system]]
; test phases: Different phases of test execution defined by Fuego: pre_test, build, deploy, test_run, get_testlog, test_processing, post_test. For a description of phases see: [[Architecture#fuego_test_phases]]
; test program: a program that runs on the target to execute a test and output the results. This can be a compiled program or a shell script (in which case the build step is empty)
; test run: this is a single instance of a test execution, containing logs and other information about the run. This is referred to in Jenkins as a 'build'.
; test script: the shell script that interfaces between the fuego core system and a test program. The script declares a tarfile, and functions to build, deploy and run the test. The test script runs on the host. This is also called the 'base test script'. For details about the environment that a script runs in or the functions it may call, see [[Variables]], [[Core interfaces]], and [[Test Script APIs]]
; test variable: This is the name of a variable available to the a test during it's execution. See [[Test variables]]
; test variable script: this is a script generated at test execution time by the overlay generator. It is generated when the test script (base test script) is executed, and environment variables are used to source various files and inputs for the final script. It is called 'prolog.sh'
; TOOLCHAIN: defines the toolchain or SDK for the device. This is used to select a set of environment variables to define and configure the toolchain for building programs for the intended test target.
; tools.sh: file containing the definition of toolchain variables for the different platforms installed in the container (and supported by the test environment) See [[tools.sh]] for details.
|