Release_1.5_Notes 

Fuego wiki

Login or create account

navigation
search
toolbox
  article   edit this page   history  

Release 1.5 Notes

Contents:

These are the Release Notes for the Fuego 1.5 release, which we are calling "Flame".

Major changes [edit section]

The major changes to Fuego in this release are as follows:
  • Added a new "batch test" feature
    • Added ability to manually group tests by batch-id, from the command line
  • Added some new functions to core, for use by test programs:
  • Simplified directory layout for the Fuego system
  • Changed the default port to 8090, from 8080
  • New options to install.sh
    • Ability to specify the TCP port for the Jenkins server
    • Ability to build a docker container that does not include a Jenkins server
    • Ability to build a docker container without using the docker image cache
  • Ability to install Fuego directly to a host machine (not inside a docker container)
  • Upgraded Fuego base Linux distro to Debian stretch
  • Upgraded Jenkins version to 2.164.1
  • Added many new tests
  • Added prototype support for sending results to a Squad and fserver back ends
  • Added new transport: ssh2serial - for ssh to a control board to which the target board is connected by serial port
  • Added prototype support for board power management via ftc
  • Fixed output from serio, to support live update of the console log during a command

Change details [edit section]

New format for batch tests [edit section]

In previous versions of Fuego, testplans were used for running multiple Fuego tests at the same time. Fuego 1.5 introduces a new feature called "batch tests" that implements this functionality, and is intended to replace the testplan functionality.

See Using Batch Tests for a description of batch tests, and how to use them and write new ones.

Ability to manually group tests by batch-id, from the command line [edit section]

Even if tests are not executed as part of a batch test, it is sometimes useful to group multiple tests together. A user can manually set the batch-id for a test, by setting an environment variable FUEGO_BATCH_ID prior to calling 'ftc run-test'. If you use the same batch-id for multiple tests, then you can filter the results by that batch-id later, to see those test results grouped together in a report. See the command line help for 'ftc run-test' for information on specifying a batch_id in a report. (Or see Using Batch Tests, which also explains this.)

New test functions [edit section]

A few new functions were added to the Fuego test core, for use with batch tests. See their respective documentation pages for more information:

Simplified directory layout [edit section]

In the Fuego 1.5 release, the directory layout of the Fuego system was changed. This was done to simplify the directory structure, and to reduce some steps required to install Fuego.

Specifically:

  • fuego-core is now placed inside the top-level 'fuego' directory
  • the 'engine' directory was removed in fuego-core

The use of 'engine' in directory paths under fuego-core is now deprecated, and new tests should avoid referencing that directory.

Please note that all tests that come with the Fuego system (ie that are in the fuego-core repository) have already been converted to remove references to the 'engine' directory. However, a symlink was added to the fuego-core directory to support old tests that reference the 'engine' directory.

New default Jenkins server port [edit section]

This release changes the default port for the Jenkins server to 8090. That is, when running with a default installation of Fuego, you should use the following URL to access the Jenkins server interface:

This was done to make it less likely for Fuego to collide with other networked programs. Specifically, if a machine already has an installation of Jenkins using port 8080, a new, default installation of Fuego will not conflict with this.

More install options [edit section]

For a complete list of install.sh options, run install.sh --help

User configurable TCP port [edit section]

It is now possible to specify the TCP port for Jenkins. By default the Fuego uses TCP/IP port 8090, but this can be changed to another port so that multiple instances of Fuego can be run simultaneously, or to avoid a conflict with a service already using port 8090 on your host machine.

To use a different port than 8090 for Jenkins, specify it after the image name on the command line when you run install.sh. Note that this means that you must specify a Docker image name in order to specify a non-default port. For example:

  • install.sh fuego 7777 would install Fuego, with an docker image name of 'fuego', a docker container name of 'fuego-container', and with Jenkins configured to run on port 7777

New --nojenkins option [edit section]

It is now possible to install Fuego into a docker container that omits the Jenkins user interface. Some Fuego users have their own front-ends and back-ends, and don't need Jenkins. In Fuego version 1.5, install.sh now supports an option '--nojenkins' which produces a docker container to run Fuego without the Jenkins server. This reduces the overhead of the docker container by quite a bit, for those users.

New --no-cache option [edit section]

Sometimes, docker image cache data provides stale data for a Fuego docker container build. You can force Fuego to build a container from a clean slate by specifying the '--no-cache' option on the install.sh command line.

Ability to install Fuego without a container [edit section]

Usually, for security and test reproducibility reasons, Fuego is executed inside a docker container on your host machine. That is, by default installing Fuego will create a docker container using all the software that is needed for Fuego's tests.

However, in some configurations it is desirable to execute Fuego directly on a host machine (not inside a docker container). A user may have a dedicated machine, or they may want to avoid the overhead of running a docker container.

Several changes were made to support running Fuego (and specifically 'ftc') outside of a docker container. There is a separate install script, called 'install-debian.sh' that is used to install the Fuego system onto a Debian-based Linux distribution.

Use the script ./install-debian.sh in the top-level fuego directory, instead of ./install.sh, to install Fuego directly into your host machine. Please note that this is not advised unless you know exactly what you are doing. In this configuration, Fuego will likely not be able to manage host-side test dependencies for you correctly.

Note that Fuego tests execute bash instruction sequences on the host. So there is a danger when running tests from unknown third parties that they will execute something on your test host that breaches the security.

Upgraded Fuego base Linux distro to Debian stretch [edit section]

The Fuego base distribution of Linux (inside the Fuego container) is now Debian 'Stretch'. This is an update from Debian Jessie, used in previous versions of Fuego.

Also, the 'slim' version of the docker image for this distribution is used, in order to save filesystem space (the 'slim' docker image is much smaller than the 'standard' Debian Stretch image).

Upgraded Jenkins version to 2.164.1 [edit section]

Fuego now utilizes Jenkins version 2.164.1 for the interface server (front-end and back-end) of the system. This more recent version of Jenkins supports recently updated plugins and security fixes for the system.

Added new Tests [edit section]

Here is a list of new tests that were added in the v1.5 release:

Distribution functionality tests:

  • Functional.atmtcp
  • Functional.brctl
  • Functional.iperf3_server
  • Functional.ipmi
  • Functional.libxml
  • Functional.module_init_tools
  • Functional.multipathd
  • Functional.nscd
  • Functional.openct
  • Functional.openhpid
  • Functional.samba
  • Functional.vconfig

Fuego self-tests:

  • Functional.fuego_check_need_module
  • Functional.fuego_hello_fail_check

Tests using other system's tests:

  • Functional.linaro
  • Functional.ptest

Batch tests:

  • Functional.batch_bc
  • Functional.batch_default
  • Functional.batch_fuego_selftest
  • Functional.batch_hello
  • Functional.batch_nested_test
  • Functional.batch_smoketest

Support for results management backends (prototype) [edit section]

Normally, Fuego results can be examined using one of two methods:
  • the Jenkins web interface
  • the Fuego ftc command line tool

However, this version of Fuego adds prototype support for sending Fuego test results to different results processing backends.

Two that are supported are the "Squad" backend, and the "fserver" backend. For both of these systems, you use the 'ftc put-run' command to transfer results for a run to the backend server.

See the following pages for more information:

NOTE: please note that support for external results management backends is in prototype form. This is not a fully supported Fuego feature yet, and aspects of this feature may change in future releases.

Added new specialized transport ssh2serial [edit section]

This release of Fuego adds a new specialized transport called 'ssh2serial'. This transport uses an intermediary computer between the host machine and the target board to perform operations on the board during a test. The intermediary machine is also referred to as the "proxy host".

Here is a diagram of the setup: 

  +--------+
  |        |  ssh  +---------------+         +--------------+
  | Fuego  |<----->| intermediary  | serial  |    board     |
  |  host  |       |   machine     |<------->|  under test  |
  |        |       | (proxy host)  |         +--------------+
  |        |       +---------------+
  +--------+

In order for this to work, you must install 'serio' on the intermediary machine. Also, the intermediary machine needs python (required to run serio tools). Specifically, the commands sercp and sersh should be placed in the directory /usr/local/bin on the intermediary machine.

Also, you need to provide the following variables to Fuego in the configuration file for the target board (ie fuego-ro/boards/<board>.conf):

  • TRANSPORT=ssh2serial - indicates to Fuego to use the ssh2serial transport
  • IPADDR - specifies the address of the proxy host
  • SSH_PORT (optional) - port for SSH operations on the proxy host (if non-default)
  • S2S_LOGIN (optional) - account name on proxy host that Fuego can use to connect to it
  • S2S_PASSWORD (optional) - password on proxy host for Fuego to connect to it
  • SSH_KEY (optional) - if specified, indicates the key file with authentication tokens for Fuego to use with the proxy host
  • SERIAL - serial device on proxy host where board is connected
  • BAUD (optional) - baud rate of serial connection between proxy host and board
  • LOGIN - account name on board
  • PASSWORD= (optional) - password of account on the board

You can either use a LOGIN/PASSWORD combination for access to the proxy host, or use an SSH_KEY file.

A staging area is created under /tmp on the proxy host where files are placed temporarily during transfer to and from the board. There must be sufficient filesystem space available to transfer whatever files any test may require (which may be big, for something like LTP (unless your configuration of LTP uses a pre-installed LTP binaries)).

Miscelaneous [edit section]

  • Fixed output from serio, to support live update of the console log during a command

undocumented changes [edit section]

  • Added a "null.dist" distribution overlay file (for testing non-Linux systems)
  • Refactored the container build scripts

TBWiki engine 1.8.3 by Tim Bird