Sometimes a test may fail for reasons that are hard to determine.

This page describes how Jenkins launches the shell script to execute fuego, and why you might see unexpected failures.

Result [edit section]

When Jenkins executes a Free-style job (such as Fuego jobs), it runs the sequence of commands in the test's "Execute shell, Command" section.

Often, this is a single line that sources the script for the test from $FUEGO_CORE/engine/tests/$TESTDIR/$TESTNAME.sh

Jenkins executes the sequence of commands by calling the shell with "/bin/sh -xe". The -x will cause the shell to print each command as it is executed (prefixing nested function calls with multiple '+'s to show the call depth. The -e will cause the shell to immediately when any command fails (that is, when any command exits with a non-zero exit code).

Most fuego test scripts are structured to define test functions for the different test phases, and then source the fuego shell routines for that that style of test. That is, the last line of a functional test script is usually ". $FUEGO_CORE/engine/scripts/functional.sh"

The last function called during test processing is test_processing(), which is defined in the test script, and for functional tests usually consists of one or more calls to log_compare.

It is the exit code from this last function that is the exit code for the entire script, and which results in the success or failure (as noted by Jenkins) of the entire test.

Allowing sub-step failures [edit section]

Sometimes there are operations that a test script performs that don't matter for the results of the test. For example, there might be some check or cleanup operation that is unrelated to the actual test being performed.

You can allow some of the steps in the test script to fail, and avoid erroring out the whole test, by using "set +e" in your test script.

Put "set +e" before any operations whose results are not important for the outcome of the test. You should put "set -e" after such operations.

Additional information [edit section]

See this question on StackOverflow for more information and ideas about this: http://stackoverflow.com/questions/22814559/how-when-does-execute-shell-mark-a-build-as-failure-in-jenkins