Release 1.1 (Blaze) Notes
Although this release is numbered as a "minor" release, it has some major changes in functionality. Here are some things to look out for in converting to this release:
- Changes to Jenkins (or the Jenkins/Fuego interface)
- Changes to the directory layout
- Changes to the script system
- New transports
- Additional tests
- Fuego now uses a much more recent version of Jenkins (v2.32.1)
- It uses the default Jenkins interface.
- Very few plugins are now required by Fuego itself (only 3)
- If you are unfamiliar with Jenkins, it is much easier to find information about the default Jenkins interface, than one customized with lots of plugins.
- If you want to customize this interface you still can, by selecting and configuring your own set of plugins.
- Nodes are added with a command line tool, rather than in the interface.
- Creating a node is simpler. See 'ftc add-nodes'
- Jobs are added with a command line tool, rather than in the interface.
- You can create a whole set of jobs based on a testplan, at one time
- You can create jobs for different specs.
- This makes it unnecessary to store Jenkins config.xml files for test jobs.
- This means that Fuego is more independent of Jenkins versions.
- Variables to control tests are now edited in the command block for the job, instead of via user interface elements
- e.g. Rebuild, Reboot, Test_PreCleanup, Test_PostCleanup, FUEGO_DEBUG
- This eliminates the need for some Jenkins plugins
- Build pages now shows links to the Fuego test data
- The post_test phase is now implemented by the script system itself, instead of being a Jenkins post build action.
For ease of development, Fuego directories are now expressed in 3 sets of directories inside the container:
In particular the script system and test directories (in fuego-core) are now visible outside the docker container. This means it is much easier to add new tests, specs, or testplans, or edit the fuego scripts themselves.
Here are some specific directories that have moved locations:
- /userdata/conf/boards is now in /fuego-ro/boards
- /userdata/toolchains is now in /fuego-ro/toolchains
- /userdata/logs is now in /fuego-rw/logs
- /home/jenkins/fuego/engine is now in /fuego-core/engine
- <fuego-core>/engine/scripts/overlays/specs - is now gone
- spec files are now placed in their corresponding test directories
- <fuego-core>/engine/jobs - has now been removed
- Jenkins jobs files are now created using 'ftc add-jobs'
- /userdata/logs/tests.info is now gone
- Benchmark metrics are now placed in their corresponding test directories
The most visible ones are those that used to be in board files, and formerly in Jenkins jobs. Here's what happened to these:
- in the Fuego board file:
- FUEGO_HOME is now BOARD_TESTDIR
- DISTRIB is used to declare the distribution used on a board
- This used to be in the Jenkins node file for a board
- It is now defined in the board file.
- It is now declared without a directory name:
- instead of "distrib/nologread.dist", just declare "nologread.dist"
- BOARD_OVERLAY - is no longer used (or needed)
- The Jenkins NODE_NAME is used to determine the board file to use
The following internal variables are removed or changed. These should not be used by any tests, but if they are, here are the equivalents.
- FUEGO_ENGINE_PATH - is no longer used - use $FUEGO_CORE/engine
- FUEGO_SCRIPTS_PATH - is no longer used - use $FUEGO_CORE/engine/scripts
- FUEGO_PARSER_PATH - is no longer used - use $FUEGO_CORE/engine/parser
- FUEGO_TESTS_PATH - is no longer used - use $FUEGO_CORE/engine/tests
- FUEGO_LOG_PATH - is no longer used - use $FUEGO_RW/logs
The only one of these that should appear in a test script should really be $FUEGO_CORE/engine/scripts, to source the <test_type>.sh script. (e.g. functional.sh, benchmark.sh, stress.sh).
The "ttc" transport is mainly of interest to Sony developers who are using the "ttc" tool to manage their board farms. (Actually, this was introduced in Fuego version 1.0.9, but it is supported in the 1.1 release).
The "serial" transport is for boards that have no networking capability, and are only connected to the host via a serial port.
Support for both of these transports is experimental in this release. There are known bugs with both of them. However, some tests can run with these, and it was deemed important to get the functionality into the hands of interested Fuego users.
Here are the Fuego functionality tests that were added:
- transport These test various aspects of the Fuego system itself, or of the configuration of a board in the Fuego system.
Here is a partial list of other tests that were added:
- voice processing
- network tool iptables,libpcap and iputils.
- C++ lib boost
Several existing tests were improved. For example functionality was added to the LTP test (Linux Test Project test), to support creating an Excel spreadsheet of results of the individual LTP sub-tests.
- Support for target-side temp directories that are erased between reboots
- Better support for handling board reboots
- Handling of proxies during the Docker container build
- Change in the pre-processor declaration for the Debian ARM cross-compilers
- This allows many more tests to use this cross-compiler, which has it's own install script in Fuego
The features that are candidates for the 1.2 release are:
- completion of support for all required AGL features (the AGL-JTA unfork)
- LAVA integration
- test reports
- unified output format - convert all test results to json format
- test dependency system
- test server system
- test package system
- additional tests
- some CVE-related tests
- especially a year-2038 test