Contents: Docker objects docker image docker container Fuego objects Fuego board Fuego test test elements table Fuego run Fuego testplan Jenkins objects Jenkins node Jenkins job Jenkins build

Here are details about the elements of Docker, Fuego and Jenkins that make up the objects in the system:

Here is a list of objects:

• Docker image
• Docker container
• Fuego board
• Fuego test
• Fuego testplan
• Fuego run
• Jenkins node - references a Fuego board
• Jenkins job - references a Fuego board and test
• Jenkins build - references a Fuego run

Docker objects [edit section]

docker image [edit section]

A docker image is built when Fuego is installed. It contains a Debian distribution, and tools, libraries, and packages needed to perform test execution and support the Fuego system.

docker container [edit section]

This is runtime instance of the image, that has been instantiated and has persistent state. The container is the object that is actually executed by docker, and which Jenkins and Fuego run inside of, in order to perform tests.

Fuego objects [edit section]

Fuego board [edit section]

A board is defined by:

• fuego board file: /fuego-ro/boards/<name>.board
  • defines: IPADDR, SSH_PORT, LOGIN, BOARD_TESTDIR, and many more
  • defines: PLATFORM
  • defines: DISTRIB
  • defines: TRANSPORT
• toolchain file: /fuego-ro/toolchains/<PLATFORM>-tools.sh
  • defines: AR, ARCH, CC, and many other variables for the toolchain

Also called 'Device' by Jenkins, and 'NODE_NAME' by Jenkins

Fuego test [edit section]

A test is defined by:

• fuego test files:
  • in /fuego-core/engine/tests/<test_name>
    • <test_base_name>.sh
    • tarball
    • <test_name>.spec - see Test Specs and Plans
    • test p/n logs - see Log files
    • reference.log
    • parser.py

test elements table [edit section]

Here is the same information in table form

The "test directory" is /fuego-core/engine/tests/{test_name}

A test consists of the following files or items:

File or item  ^	format  ^	location  ^	description  ^	test type  ^
reference.log	Fuego-specific	test directory	Has the threshold values and comparison operators for benchmark metrics measured by the test	benchmarks only
metrics.json	JSON	test directory	provides the names of metrics to be plotted for this benchmark	benchmarks only
test plan(s)	JSON	/fuego-core/engine/overlays/testplans/testplan_default.json	Has the testplan(s) for this test	all
test spec	JSON	test directory	Has groups of variables (and their values) that can be used with this test	all
test.yaml	YAML	test directory	Has the "manifest" for packaging a test	all
patches	patch format	test directory	Zero or more patches to customize the test program (applied during the unpack phase	all
parser.py	python	test directory	Python program to parse benchmark metrics out of the log, and provide a dictionary to the Fuego plotter	benchmarks only
base script	shell script	test directory	Is the shell script that implements the different test phases in Fuego	all
tarfile	tar format	test directory	Has the source code for the test program	all
p/n logs	text	test directory	Are logs with the results (positive or negative) parsed out, for determination of test pass/fail	functional only

Query

(See also Adding a test, Benchmark parser notes, parser.py, reference.log

Fuego run [edit section]

A test run is defined by:

• fuego log files:
  • /fuego-rw/logs/<test_name>/<board>.<spec>.<build_id>.<build_number>/
    • testlog.txt
    • syslog.before.txt
    • syslog.after.txt
    • consolelog.txt
    • testlog.p.txt, testlog.n.txt
    • (coming soon: fres.json)
• benchmark results:
  • /fuego-rw/logs/<test_name>
    • plot.data
    • plot.png
    • <testname>.info.json
    • <testname>.<metric1>.json
      • with a file for each metric

Fuego testplan [edit section]

• This file indicates a set of tests to run, and associated test variants (test specs) and other parameters (build variables from the spec)
  • /fuego-core/engine/overlays/testplans/testplan_<name>.json

See Test Specs and Plans for more information.

Jenkins objects [edit section]

Jenkins node [edit section]

• This is the Jenkins object to represent a Fuego board
  • /var/lib/jenkins/nodes/<node_name>/config.xml

Jenkins job [edit section]

• This is the Jenkins object to represent a unit of test execution
• It refers to a node, spec, and test
  • The job name is composed of: <node_name>.<spec>.<test_type>.<test_name>
• jenkins job file:
  • /var/lib/jenkins/jobs/<job_name>/config.xml - expected to be static
    • provides form for: 'Reboot', 'Rebuild', 'Target_Cleanup'
    • specifies shell command sequence to cause build
      • this references the script for the test, at:
        • $FUEGO_CORE/engine/tests/$TESTDIR/$TEST_NAME.sh
  • /var/lib/jenkins/jobs/<job_name>/nextBuildNumber - changes for each run

Jenkins build [edit section]

• jenkins files:
  • /var/lib/jenkins/jobs/<job_name>/builds/<build_id>/
    • build.xml
    • changelog.xml
    • log - this is the console log