Contents: Major changes Changes to Jenkins Changes to directory layout Changes to the script system New transports Additional tests Bugfixes Roadmap

These are the Release Notes for the Fuego 1.1 release, which we are calling "Blaze".

Although this release is numbered as a "minor" release, it has some major changes in functionality. Here are some things to look out for in converting to this release:

Major changes [edit section]

The major changes to Fuego in this release can be categorized as follows:

• Changes to Jenkins (or the Jenkins/Fuego interface)
• Changes to the directory layout
• Changes to the script system
• New transports
• Additional tests

Changes to Jenkins [edit section]

There were major changes to

• Fuego now uses a much more recent version of Jenkins (v2.32.1)
• It uses the default Jenkins interface.
  • Very few plugins are now required by Fuego itself (only 3)
  • If you are unfamiliar with Jenkins, it is much easier to find information about the default Jenkins interface, than one customized with lots of plugins.
  • If you want to customize this interface you still can, by selecting and configuring your own set of plugins.
• Nodes are added with a command line tool, rather than in the interface.
  • Creating a node is simpler. See 'ftc add-nodes'
• Jobs are added with a command line tool, rather than in the interface.
  • You can create a whole set of jobs based on a testplan, at one time
  • You can create jobs for different specs.
  • This makes it unnecessary to store Jenkins config.xml files for test jobs.
    • This means that Fuego is more independent of Jenkins versions.
• Variables to control tests are now edited in the command block for the job, instead of via user interface elements
  • e.g. Rebuild, Reboot, Test_PreCleanup, Test_PostCleanup, FUEGO_DEBUG
  • This eliminates the need for some Jenkins plugins
• Build pages now shows links to the Fuego test data
• The post_test phase is now implemented by the script system itself, instead of being a Jenkins post build action.

Changes to directory layout [edit section]

Almost all of the Fuego directories have changed locations. They used to be expressed as a set of directories under /userdata and /home/jenkins.

For ease of development, Fuego directories are now expressed in 3 sets of directories inside the container:

• /fuego-ro
• /fuego-rw
• /fuego-core

In particular the script system and test directories (in fuego-core) are now visible outside the docker container. This means it is much easier to add new tests, specs, or testplans, or edit the fuego scripts themselves.

Here are some specific directories that have moved locations:

• /userdata/conf/boards is now in /fuego-ro/boards
• /userdata/toolchains is now in /fuego-ro/toolchains
• /userdata/logs is now in /fuego-rw/logs
• /home/jenkins/fuego/engine is now in /fuego-core/engine
• <fuego-core>/engine/scripts/overlays/specs - is now gone
  • spec files are now placed in their corresponding test directories
• <fuego-core>/engine/jobs - has now been removed
  • Jenkins jobs files are now created using 'ftc add-jobs'
• /userdata/logs/tests.info is now gone
  • Benchmark metrics are now placed in their corresponding test directories

Changes to the script system [edit section]

A number of variables have been changed in the Fuego script system.

The most visible ones are those that used to be in board files, and formerly in Jenkins jobs. Here's what happened to these:

• in the Fuego board file:
  • FUEGO_HOME is now BOARD_TESTDIR
  • DISTRIB is used to declare the distribution used on a board
    • This used to be in the Jenkins node file for a board
    • It is now defined in the board file.
    • It is now declared without a directory name:
      • instead of "distrib/nologread.dist", just declare "nologread.dist"
• BOARD_OVERLAY - is no longer used (or needed)
  • The Jenkins NODE_NAME is used to determine the board file to use

The following internal variables are removed or changed. These should not be used by any tests, but if they are, here are the equivalents.

• FUEGO_ENGINE_PATH - is no longer used - use $FUEGO_CORE/engine
• FUEGO_SCRIPTS_PATH - is no longer used - use $FUEGO_CORE/engine/scripts
• FUEGO_PARSER_PATH - is no longer used - use $FUEGO_CORE/engine/parser
• FUEGO_TESTS_PATH - is no longer used - use $FUEGO_CORE/engine/tests
• FUEGO_LOG_PATH - is no longer used - use $FUEGO_RW/logs

The only one of these that should appear in a test script should really be $FUEGO_CORE/engine/scripts, to source the <test_type>.sh script. (e.g. functional.sh, benchmark.sh, stress.sh).

New transports [edit section]

Two new transports have been added to this release:

• ttc
• serial

The "ttc" transport is mainly of interest to Sony developers who are using the "ttc" tool to manage their board farms. (Actually, this was introduced in Fuego version 1.0.9, but it is supported in the 1.1 release).

The "serial" transport is for boards that have no networking capability, and are only connected to the host via a serial port.

Support for both of these transports is experimental in this release. There are known bugs with both of them. However, some tests can run with these, and it was deemed important to get the functionality into the hands of interested Fuego users.

Additional tests [edit section]

A number of new tests have been added, including some for testing Fuego functionality itself. A set of tests were added from the AGL-JTA project. These latter tests need more work to preserve reporting capabilities that were available in JTA. But this release provides a baseline of operation for these tests, to build upon for the next release.

Here are the Fuego functionality tests that were added:

• check_plots
• abort
• board_check
• test_phases
• transport These test various aspects of the Fuego system itself, or of the configuration of a board in the Fuego system.

Here is a partial list of other tests that were added:

• xorg-macros
• tiff
• thrift
• protobuf
• pppd
• pixman
• net-tools
• neon
• lwip
• libwebsocket
• libtar
• librsv
• voice processing
• kmod
• network tool iptables,libpcap and iputils.
• imagemagick.
• hciattach
• glibc
• giflib
• fuse
• fixesproto
• curl
• libcroco
• bsdiff
• C++ lib boost

Several existing tests were improved. For example functionality was added to the LTP test (Linux Test Project test), to support creating an Excel spreadsheet of results of the individual LTP sub-tests.

Bugfixes [edit section]

There were numerous bugfixes in this release. Some of the major ones were:

• Support for target-side temp directories that are erased between reboots
• Better support for handling board reboots
• Handling of proxies during the Docker container build
• Change in the pre-processor declaration for the Debian ARM cross-compilers
  • This allows many more tests to use this cross-compiler, which has it's own install script in Fuego

Roadmap [edit section]

This release took a long time - almost 6 months. We would like to have shorter releases for the next year (possibly 3 or 4-months apart). We have several important features to release and we'd like to get them to users as soon as possible.

The features that are candidates for the 1.2 release are:

• completion of support for all required AGL features (the AGL-JTA unfork)
  • LAVA integration
  • test reports
• unified output format - convert all test results to json format
• test dependency system
• test server system
• test package system
• additional tests
  • some CVE-related tests
  • especially a year-2038 test