Fuego wiki

Login or create account

Test variables in 'raw' format


= Introduction =
When Fuego executes a test, shell environment variables are used to
provide information about the test environment, test execution
parameters, communications methods and parameters, and other items.

These pieces of information are originate from numerous different places.
An initial set of test variables comes in the shell environment from
either Jenkins or ftc (depending on which one is used to invoke the test).

The information about the board being tested comes primarily from two sources:
 * the board file
 * the dynamic board variables file

Additional information comes from the testplan and test spec that are
used for this particular test run.

Test variables can be simple strings, or they may be shell functions.

When a test is run, Fuego gathers information from all these sources,
and makes them available to the test (and uses them itself) to control
test execution.

= Board file =
The board file contains static information about a board.  It is processed
by the overlay system, and the values inside it appear as variables
in the environment of a test, during test execution.

The board file resides in:
 * /fuego-ro/boards/$BOARD.board

There are a number of variables which are used by the Fuego system itself,
and there may also be variables that are used by individual tests.

== Common board variables ==
Here is a list of the variables which might be found in a board file:
 * ARCHITECTURE - specifies the architecture of the board
 * BAUD - baud rate for serial device (if using 'serial' transport)
 * BOARD_TESTDIR - directory on board where tests are executed
 * BOARD_CONTROL - the mechanism used to control board hardware (e.g. hardware reboot)
 * DISTRIB - filename of distribution overlay file (if not the default)
 * IO_TIME_SERIAL - serial port delay parameter (if using 'serial' transport)
 * IPADDR - network address of the board
 * LOGIN - specifies the user account to use for Fuego operations
 * PASSWORD - specifies the password for the user account on the board used by Fuego
 * PLATFORM - specifies the toolchain to use for the platform
 * SATA_DEV - specifies a filesystem device node (on the board) for SATA filesystem tests
 * SATA_MP - specifies a filesystem mount point (on the board) for SATA filesystem tests
 * SERIAL - serial device on host for board's serial console (if using 'serial' transport)
 * SRV_IP - network address of server endpoint, for networking tests (if not the same as the host)
 * SSH_KEY - the absolute path to key file  with ssh key for password-less ssh operations (e.g. "/fuego-ro/board/myboard_id_rsa")
 * SSH_PORT - network port of ssh daemon on board (if using ssh transport)
 * TRANSPORT - this specifies the transport to use with the target
 * USB_DEV - specifies a filesystem device node (on the board) for USB filesystem tests
 * USB_MP - specifies a filesystem mount point (on the board) for USB filesystem tests

See [[Adding a board]] for more details about these variables.

A board may also have additional variables, including variables that
are used for results evaluation for specific tests.

= Overlay system =
The overlay system gathers variables from several places, and puts
them all together into a single file which is then source'ed into the
running test's environment.

It takes information from:
 * the board files (both static and dynamic)
 * the testplan
 * the test spec
 * the overlay area
and combines them all, using a set of priorities, into a single
file called "", which is then source'ed into the running
shell environment of the Fuego test being executed.

The overlay system is described in greater detail here:

= Dynamic variables =
Dynamic board variables are test variables that are defined on a per-board
basis, and can be modified and managed under program control.

Dynamic variables allow the Fuego system, a test, or a user to store
information that can be used by tests.  This essentially
creates an information cache about the board, that can be both
manually and programmatically generated and managed.

The information that needs to be held for a particular board depends on the tests that are installed in the system. Thus the system needs to support
ad-hoc collections of variables.  Just putting everything into the static board
file would not scale, as the number of tests increases.

''Note: the LAVA test framework has a similar concept called a "board dictionary".''

One use case for this to have a "board setup" test, that scans for lots of 
different items, and populates the dynamic variables with values that
are used by other tests.  Some items that are useful to know about a board
take time to discover (using e.g. 'find' on the target board), and using
a board dynamic variable can help reduce the time required to check these items.

The board dynamic variables are kept in the file:
 * /fuego-rw/boards/$BOARD.vars

These variables are included in the test by the overlay generator.

== Commands for interacting with dynamic variables ==
A user or a test can manipulate a board dynamic variable using the ftc command.
The following commands can be used to set, query and delete variables:
 * '''tc query-board''' - to see test variables (that are board-related)
 * '''ftc set-var''' - to add or update a dynamic variable
 * '''ftc delete-var''' - to delete a dynamic variable

=== ftc query-board ===
'ftc query-board' is used to view the variables associated with a Fuego board.
You can use the command to see all the variables, or just a single variable.

Note that 'ftc query-board' shows the variables for a test that come from both the board file and board dynamic variables file (that is, both 'static' board
variables and dynamic variables).  It does not show variables which come from testplans or spec files, as those are specific to a test.

The usage is:
 * ftc query-board <board> [-n <VARIABLE>]

 $ ftc query-board myboard
 $ ftc query-board myboard -n PROGRAM_BC

The first example would show all board variables, including functions.
The second example would show only the variable PROGRAM_BC, if it existed, for board 'myboard'.

=== ftc set-var ===
'ftc set-var' allows setting or updating the value of a board dynamic variable.

The usage is:
 * ftc set-var <board> <VARIABLE>=<value>

By convention, variable names are all uppercase, and function names are
lowercase, with words separated by underscores.

 $ ftc set-var PROGRAM_BC=/usr/bin/bc

=== ftc delete-var ===
'ftc delete-var' removes a variable from the dynamic variables file.

 $ ftc delete-var PROGRAM_BC

== Example usage ==
Functional.fuego_board_check could detect the path for the 'foo' binary,
(e.g. is_on_target foo PROGRAM_FOO)
and call 'ftc set-var $NODE_NAME PROGRAM_FOO=$PROGRAM_FOO'.
This would stay persistently defined as a test variable, so other
tests could use $PROGRAM_FOO (with assert_defines, or in
'report' or 'cmd' function calls.)

== Example Dynamic variables ==
Here are some examples of variables that can be kept as dynamic variables,
rather than static variables from the board file:
 * SATA_DEV = Linux device node for SATA file system tests
 * SATA_MP = Linux mount point for SATA file system tests
 * LTP_OPEN_POSIX_SUBTEST_COUNT_POS = expected number of pass results for LTP OpenPosix test
 * LTP_OPEN_POSIX_SUBTEST_COUNT_NEG = expected number of fail results for LTP OpenPosix test
 * PROGRAM_BC = path to 'bc' program on the target board
 * MAX_REBOOT_RETRIES = number of retries to use when rebooting a board

TBWiki engine 1.8.2 by Tim Bird