Release_1.4_Notes 

Fuego wiki

Login or create account

Release 1.4 Notes

Note: This page is under construction during the 1.4 release development cycle.

These are the Release Notes for the Fuego 1.4 release, which we are calling "Ember".

Major changes [edit section]

The major changes to Fuego in this release are as follows:
  • added of clitest to Fuego distribution in the docker container (doc'ed)
  • added fuego_board_function_lib.sh
  • new functions for test programs:
  • added new make_cache feature, to pre-build test binaries
  • added (and used) FUEGO_LOGLEVELS to replace FUEGO_DEBUG
    • added dprint, vprint, iprint, wprint, eprint routines
  • did major cleanup of console log verbosity
  • added new 'snapshot' phase of test execution
  • converted Jenkins jobs to call ftc run-test
  • added support for dynamic variables
  • improved support for (still prototype) central server (fserver.py)
    • added support for server polling for remote job execution
    • reworked support for fuego server operations in ftc
  • simplified install and start of container
  • PLATFORM was renamed to TOOLCHAIN
  • added "ftc reserve-resource" and "ftc release-resource"

  • Test maintenance:
    • glib, glibc, iperf, more
    • build-out of Functional.busybox test

  • new tests:
    • tar, bind, etc.

Change details [edit section]

clitest [edit section]

clitest is a handy shell script for testing the results of command line operations. It is written purely in POSIX, so it can be used for both host-side and board-side testing.

It is installed in /usr/local/bin in the Fuego docker container, and is available for any tests that would like to use it. To use it on the board being tested, copy it to the board during the test's 'deploy' phase (in test_deploy in the test's fuego_test.sh file), with a command line so:

  • put /usr/local/bin/clitest $BOARD_TESTDIR/fuego.$TESTDIR

Then use it in a command one the board (usually 'report'), like so:

  • report "cd $BOARD_TESTDIR/fuego.$TESTDIR ; ./clitest --prefix TAB cli-command-file"

added fuego_board_function_lib.sh [edit section]

We have started accumulating some utility functions for performing certain operations in a distribution-independent way into a library of shell functions for use on target. Use of this library is completely optional for a test. The library is intended to make certain common operations available to board-side shell scripts.

If might be interested in using this library if you have a board-side shell script that needs to:

  • 1) detect the type of init manager (proc 1) running on the board
  • 2) detect the type of logger service running on the board
  • 3) start or stop services on the board, as part of your test

To use this library, your fuego_test.sh script should copy the library script to the board in the test_deploy function, and use it from a board-side shell script (during the 'run' phase of the test).

See fuego_board_function_lib.sh for more details.

log_this [edit section]

Some tests need to get information and data from host-side operations, that needs to be reported and analyzed by Fuego.

The new log_this function captures the output of commands executed on the host, and puts it (ultimately) into the test log for a run. Any command executed with "log_this" is saved during test execution, and placed in the final testlog.txt, after any board-side log data (from report and report_append) commands.

There are a few tests (especially Fuego self-tests) that use this feature to avoid an awkward sequence of push-to-target, and report-cat, to get log data from the host into the testlog.

report_live function [edit section]

A new report_live function is available to capture data directly from the test program on the target onto the host. The normal report function saves the test program output to a log file, which is retrieved after the test completes. For some tests (e.g. tests that reboot the machine, or kexec into a new kernel), it is not possible to save the data for the whole test into a log on the target board.

The report_live function saves the data synchronously to the host, and puts it into the test log during post-processing.

assert_has_program function [edit section]

See assert_has_program FIXTHIS - need more docs for assert_has_program

is_abs_path function [edit section]

is_abs_path is a small helper function added to the Fuego core library, for detecting if a path is absolute or not.

make_cache [edit section]

FIXTHIS - document make_cache.sh feature

It is now possible to create a cache of pre-built test binary packages. These packages can be used to manually install the test program on the board. A new phase was added 'm' for makepkg. if phase makepkg is specified, then deploy is altered to put the materials into /fuego-rw/stage/fuego.<testname>. Then, after deployment makepkg is called to create a test program package file, in /fuego-rw/cache/$TOOLCHAIN-$TESTNAME-board-package.tar.gz

FUEGO_LOGLEVELS feature [edit section]

  • added (and used) FUEGO_LOGLEVELS to replace FUEGO_DEBUG
    • added dprint, vprint, iprint, wprint, eprint routines

log verbosity [edit section]

  • did major cleanup of console log verbosity

snapshot feature [edit section]

  • added new 'snapshot' phase of test execution
  • there's a new phase "snapshot" (ftc -p 's')
  • executes test_snapshot function, if defined in fuego_test.sh
    • if no test_snapshot() function is defined, then does default (ov_rootfs_state)
  • test_snapshot() should put data into $SNAPSHOT_FILENAME

New file created for every test run: machine_snapshot.txt (as long as it doesn't error out before the test_run phase).

Is saved in log directory for the run (under /fuego-rw/logs).

Can be customized by a test to grab whatever information from the board might be useful for diagnostic use after the run completes.

ftc run-test [edit section]

  • converted Jenkins jobs to call ftc run-test

dynamic variables [edit section]

  • added support for dynamic test variables
    • ftc run-test --dynamic-vars "FOO=blah"
    • spec.json with dynamic vars is created in log directory
    • allows for scripted variations of test runs (e.g. bisecting)

fuego object server [edit section]

  • improved support for (still prototype) central server (fserver.py)
    • added support for server polling for remote job execution
    • reworked support for fuego server operations in ftc

See https://github.com/tbird20d/fserver.git

Install simplification [edit section]

  • simplified install and start of container

TOOLCHAIN board variable [edit section]

  • PLATFORM was renamed to TOOLCHAIN
    • PLATFORM variable is still supported, for backwards compatibility, but it is deprecated.

resource reservation [edit section]

  • added "ftc reserve-resource" and "ftc release-resource"

This is the start of a feature for allowing the scheduling of lab resources for a test. For example, a test may need to use the 'netperf' server during a test of networking performance. It is desirable for this server to be used by only one test at a time, in order to avoid disturbing the results of the test on one board, with packets and networking activity from another board. A test can reserve the netperf server by doing ftc reserve-resource 'netperf-server'

If a board in the test lab should be reserved, then the following should be used by the test:

  • ftc reserve-resource -b other-board

Resources are held for a fixed period of time. If another test or user attempts to reserve a resource already reserved, then the reservation will fail.

New tests in the 1.4 release [edit section]

The following new tests have been added in this release:
  • Functional.fuego_ftc_test

Test details [edit section]

Functional.fuego_ftc_test
tests various features of the ftc command-line tool, to make sure that new functionality has not broken anything.

TBWiki engine 1.8.2 by Tim Bird