{{TableOfContents}}

These are the Release Notes for the Fuego 1.5 release, which we are calling "Flame".

Major changes [edit section]

= Major changes =
The major changes to Fuego in this release are as follows:
 * Added a new "batch test" feature
   * Added ability to manually group tests by batch-id, from the command line
 * Added some new functions to core, for use by test programs:
   * [[function_run_test|run_test]]
   * [[function_allocate_next_batch_id|allocate_next_batch_id]]
 * Simplified directory layout for the Fuego system
 * Changed the default port to 8090, from 8080
 * New options to install.sh
   * Ability to specify the TCP port for the Jenkins server
   * Ability to build a docker container that does not include a Jenkins server
   * Ability to build a docker container without using the docker image cache
 * Ability to install Fuego directly to a host machine (not inside a docker container)
 * Upgraded Fuego base Linux distro to Debian stretch
 * Upgraded Jenkins version to 2.164.1
 * Added many new tests
 * Added prototype support for sending results to a Squad and fserver back ends
 * Added new transport: ssh2serial - for ssh to a control board to which the target board is connected by serial port
 * Added prototype support for board power management via ftc
 * Fixed output from serio, to support live update of the console log during a command

Change details [edit section]

= Change details =

New format for batch tests [edit section]

== New format for batch tests ==
In previous versions of Fuego, testplans were used for running multiple Fuego
tests at the same time.  Fuego 1.5 introduces a new feature called
"batch tests" that implements this functionality, and is intended to replace
the testplan functionality.

See [[Using Batch Tests]] for a description of batch tests, and how
to use them and write new ones.

Ability to manually group tests by batch-id, from the command line [edit section]

== Ability to manually group tests by batch-id, from the command line ==
Even if tests are not executed as part of a batch test, it is sometimes 
useful to group multiple tests together.  A user can manually set
the batch-id for a test, by setting an environment variable
FUEGO_BATCH_ID prior to calling 'ftc run-test'.  If you use the same 
batch-id for multiple tests, then you can filter the results by that
batch-id later, to see those test results grouped together in a report.
See the command line help for 'ftc run-test' for information on specifying
a batch_id in a report.  (Or see [[Using Batch Tests]], which also
explains this.)

New test functions [edit section]

== New test functions ==
A few new functions were added to the Fuego test core, for use with
batch tests.  See their respective documentation pages for more
information:
 * [[function_run_test|run_test]]
 * [[function_allocate_next_batch_id|allocate_next_batch_id]]

Simplified directory layout [edit section]

== Simplified directory layout ==
In the Fuego 1.5 release, the directory layout of the Fuego system
was changed.  This was done to simplify the directory structure, and
to reduce some steps required to install Fuego.

Specifically:
 * fuego-core is now placed inside the top-level 'fuego' directory
 * the 'engine' directory was removed in fuego-core

The use of 'engine' in directory paths under fuego-core is now deprecated, and new tests should avoid referencing that directory.

Please note that all tests that come with the Fuego system (ie that are in the fuego-core repository) have already been converted to remove references to
the 'engine' directory. However, a symlink was added to the fuego-core
directory to support old tests that reference the 'engine' directory.

New default Jenkins server port [edit section]

== New default Jenkins server port ==
This release changes the default port for the Jenkins server to 8090.
That is, when running with a default installation of Fuego, you should
use the following URL to access the Jenkins server interface:
 * http://localhost:8090/fuego

This was done to make it less likely for Fuego to collide with other
networked programs.  Specifically, if a machine already has an installation
of Jenkins using port 8080, a new, default installation of Fuego will
not conflict with this.

More install options [edit section]

== More install options =
For a complete list of install.sh options, run ``install.sh --help``

User configurable TCP port [edit section]

=== User configurable TCP port ===
It is now possible to specify the TCP port for Jenkins.  By default the
Fuego uses TCP/IP port 8090, but this can be changed to another port so that
multiple instances of Fuego can be run simultaneously, or to avoid a conflict
with a service already using port 8090 on your host machine.

To use a different port than 8090 for Jenkins, specify it after the
image name on the command line when you run install.sh.  Note that this means
that you must specify a Docker image name in order to specify a non-default
port.  For example:
 * install.sh fuego 7777
would install Fuego, with an docker image name of 'fuego', a docker
container name of 'fuego-container', and with Jenkins configured to run on
port 7777

New --nojenkins option [edit section]

=== New --nojenkins option ===
It is now possible to install Fuego into a docker container that omits
the Jenkins user interface.  Some Fuego users have their own front-ends
and back-ends, and don't need Jenkins.  In Fuego version 1.5, install.sh
now supports an option '--nojenkins' which produces a docker container
to run Fuego without the Jenkins server.  This reduces the overhead
of the docker container by quite a bit, for those users.

New --no-cache option [edit section]

=== New --no-cache option ===
Sometimes, docker image cache data provides stale data for a Fuego 
docker container build.  You can force Fuego to build a container
from a clean slate by specifying the '--no-cache' option on the
install.sh command line.

Ability to install Fuego without a container [edit section]

== Ability to install Fuego without a container ==
Usually, for security and test reproducibility reasons, Fuego is
executed inside a docker container on your host machine.  That is,
by default installing Fuego will create a docker container using
all the software that is needed for Fuego's tests.

However, in some configurations it is desirable to execute Fuego
directly on a host machine (not inside a docker container).
A user may have a dedicated machine, or they may want to avoid
the overhead of running a docker container.

Several changes were made to support running Fuego (and specifically 'ftc')
outside of a docker container.  There is a separate install script,
called 'install-debian.sh' that is used to install the Fuego system
onto a Debian-based Linux distribution.

Use the script ``./install-debian.sh`` in the top-level fuego directory, instead of ``./install.sh``, to install Fuego directly into your host machine.
Please note that this is not advised unless you know exactly what you are
doing.  In this configuration, Fuego will likely not be able to manage
host-side test dependencies for you correctly.

Note that Fuego tests execute bash instruction sequences on the host.
So there is a danger when running tests from unknown third parties that they will execute something on your test host that breaches the security.  

Upgraded Fuego base Linux distro to Debian stretch [edit section]

== Upgraded Fuego base Linux distro to Debian stretch ==
The Fuego base distribution of Linux (inside the Fuego container) is now
Debian 'Stretch'.  This is an update from Debian Jessie, used in previous
versions of Fuego.

Also, the 'slim' version of the docker image for this distribution is
used, in order to save filesystem space (the 'slim' docker image is
much smaller than the 'standard' Debian Stretch image).

Upgraded Jenkins version to 2.164.1 [edit section]

== Upgraded Jenkins version to 2.164.1 ==
Fuego now utilizes Jenkins version 2.164.1 for the interface server (front-end
and back-end) of the system.  This more recent version of Jenkins supports
recently updated plugins and security fixes for the system.

Added new Tests [edit section]

== Added new Tests ==
Here is a list of new tests that were added in the v1.5 release:

Distribution functionality tests:
 * Functional.atmtcp
 * Functional.brctl
 * Functional.iperf3_server
 * Functional.ipmi
 * Functional.libxml
 * Functional.module_init_tools
 * Functional.multipathd
 * Functional.nscd
 * Functional.openct
 * Functional.openhpid
 * Functional.samba
 * Functional.vconfig

Fuego self-tests:
 * Functional.fuego_check_need_module
 * Functional.fuego_hello_fail_check

Tests using other system's tests:
 * Functional.linaro
 * Functional.ptest

Batch tests:
 * Functional.batch_bc
 * Functional.batch_default
 * Functional.batch_fuego_selftest
 * Functional.batch_hello
 * Functional.batch_nested_test
 * Functional.batch_smoketest

Support for results management backends (prototype) [edit section]

== Support for results management backends (prototype) ==
Normally, Fuego results can be examined using one of two methods:
 * the Jenkins web interface
 * the Fuego ftc command line tool

However, this version of Fuego adds prototype support for sending Fuego test results to different results processing backends.

Two that are supported are the "Squad" backend, and the "fserver" backend.
For both of these systems, you use the 'ftc put-run' command to transfer
results for a run to the backend server.

See the following pages for more information:
 * [[Squad backend notes]]
 * [[Using Fuego with fserver]]

NOTE: please note that support for external results management backends is in prototype form. This is not a fully supported Fuego feature yet, and aspects of this feature may change in future releases.

''NOTE: please note that support for external results management backends is
in prototype form.  This is not a fully supported Fuego feature yet, and 
aspects of this feature may change in future releases.''

Added new specialized transport ssh2serial [edit section]

== Added new specialized transport ssh2serial ==
This release of Fuego adds a new specialized transport called 'ssh2serial'.
This transport uses an intermediary computer between the host machine and the target board to
perform operations on the board during a test. The intermediary machine is also referred
to as the "proxy host".

Here is a diagram of the setup:
{{{
  +--------+
  |        |  ssh  +---------------+         +--------------+
  | Fuego  |<----->| intermediary  | serial  |    board     |
  |  host  |       |   machine     |<------->|  under test  |
  |        |       | (proxy host)  |         +--------------+
  |        |       +---------------+
  +--------+
}}}

In order for this to work, you must install 'serio' on the intermediary
machine.  Also, the intermediary machine needs python (required to run
serio tools).  Specifically, the commands ``sercp`` and ``sersh`` should
be placed in the directory /usr/local/bin on the intermediary machine.

Also, you need to provide the following variables to Fuego in the
configuration file for the target board (ie fuego-ro/boards/<board>.conf):
 * TRANSPORT=ssh2serial - indicates to Fuego to use the ssh2serial transport
 * IPADDR - specifies the address of the proxy host
 * SSH_PORT (optional) - port for SSH operations on the proxy host (if non-default)
 * S2S_LOGIN (optional) - account name on proxy host that Fuego can use to connect to it
 * S2S_PASSWORD (optional) - password on proxy host for Fuego to connect to it
 * SSH_KEY (optional) - if specified, indicates the key file with authentication tokens for Fuego to use with the proxy host
 * SERIAL - serial device on proxy host where board is connected
 * BAUD (optional) - baud rate of serial connection between proxy host and board
 * LOGIN -  account name on board
 * PASSWORD= (optional) - password of account on the board

You can either use a LOGIN/PASSWORD combination for access to the proxy host, or
use an SSH_KEY file.

A staging area is created under /tmp on the proxy host where files are placed
temporarily during transfer to and from the board.  There must be sufficient
filesystem space available to transfer whatever files any test may require
(which may be big, for something like LTP (unless your configuration of LTP uses
a pre-installed LTP binaries)).

Miscelaneous [edit section]

== Miscelaneous ==
 * Fixed output from serio, to support live update of the console log during a command

undocumented changes [edit section]

= undocumented changes =
 * Added a "null.dist" distribution overlay file (for testing non-Linux systems)
 * Refactored the container build scripts