{{TableOfContents}}

''Note: This is a feature of Fuego that is in prototype form.  That is, 
this feature is currently under development, and may change as the code
matures and is readied for production use.  It is presented here now
to allow for experimentation and feedback.''

= Quicklink =
The main global fserver web site is here: https://fuegotest.org/cgi-bin/fserver.py

A test fserver web site is here: https://birdcloud.org/cgi-bin/fserver.py

= Introduction =
Normally, Fuego uses Jenkins for some important elements of the
test automation process and lifecycle. This includes:
 * triggering of test execution
 * test scheduling (ie board scheduling and test timeout management)
 * test results storage
 * test results presentation
 * test results notifications

Some of these elements occur before a test is executed, and some occur
after a test is executed. Fuego supports integration with external services
for some of these actions.  For example, some Fuego labs use LAVA for
board provisioning and test scheduling.

This page describes 'fserver', which is a json-oriented test object manager
designed to support Fuego objects.

Normally, Fuego stores test results on the test host (where Fuego is installed).  These results can be queried and perused (browsed, examined,
have visualizations presented) using a command line tool and via the
Jenkins web interface.

However, it is also possible to send test objects, including test packages, test requests, and test results to a different service for management and processing there.

This page documents the use of Fuego with the 'fserver' test object server.

== fserver design philosophy ==
Fserver is current designed to be a "dumb json object store".  It does not
currently (as of April 2019, version 0.2.0) do any active management
or processing of test objects.  There are 3 different objects stored currently
by fserver:
 * test objects (e.g. test definitions)
 * test requests
 * test runs (ie test results objects, or results artifacts)

It is anticipated that this list of test objects may be expanded in the future
to include additional object types, including the following (potential):
 * test reviews (including ratings)
 * build artifacts (build images)
 * test target packages
 * test report definitions
 * lab definitions
 * board definitions
 * user objects (for authentication)

Fserver acts as an external repository of test objects.  Note
that all processing logic is managed by remote nodes that interact
with the server.

For example, a user may request that a test to be executed on a board in another
lab by adding a test object to the fserver respository, and also adding a test request object to the fserver repository, with the desired
lab and board specified in the test request object.  fserver itself does
not interpret the data in the objects in any way.  If the other
lab (the one mentioned in the test request) is polling for jobs on
fserver, that lab will note that there is a job
for it to perform, and will download the job, execute it, and put a run
object (test results object) back on the fserver site.  Then, the first
site can download the data for the test and examine or manipulate the
results.

The fserver clients (the requesting host, and the Fuego host in the lab
that executes the test) are completely responsible for managing the data
in the fserver repository.  This includes updating the status of the test
request (in real time), while the job is being executed, as well as removing
test requests and run results when they are no longer needed.

An fserver acts as a web-based rendezvous point for Fuego labs to communicate
with each other, to coordinate distributed automated test activities.



= fserver installation =
You can use fserver from an external fserver installation, or set up your
own fserver server.  The fserver software is pre-installed
in the Fuego docker container when Fuego is installed, and no additional
installation is required if you decide to use that server.

Also, there is a global instance of fserver running on the Internet on
the ``fuegotest.org`` domain.  You can use this fserver for experimentation
if you'd like.  But please do not use it for production use without
first communicating with Tim Bird on the Fuego mailing list.

If you decide to install fserver directly on your host (outside the
docker container) or on another machine, you may use the following instructions
to manually install and start fserver.

== Install the fserver server software ==
To install your own instance of the fserver software (not the one included
in the Fuego docker container), do the following:
 * determine an installation directory (usually /usr/local/lib/fserver, or maybe somewhere in your home directory: ~/fserver)
 * clone the fserver software from github to that directory:
    * $ git clone https://github.com/tbird20d/fserver.git ~/fserver

The main CGI script for fserver is called fserver.py

== Using fserver with an existing web server ==
You can run fserver using an existing web server, or you can
run fserver using a standalone local web server.

If using an existing web server, put the fserver.py script in the
server's cgi-bin directory (or mark it in the web server configuration
as a CGI script.)

For many Linux distros, this would mean putting the script in
/usr/lib/cgi-bin.

Modify the variables in fserver.py to reflect the installation location
for where you installed the server source (including the fserver-data
directory).

You will also need to make sure that the web server has access
to the fserver-data directory.  Usually a command like this will
suffice:
{{{
  $ sudo chown -R www-data.www-data fserver-data
}}}

You will also need to provide a link so that the web server can access the
fserver-data directories, so that users can download objects (such as
json, yaml and package files) directly via the web server.

If your web server has its root www directory at /var/www/html, and your
fserver-data directory is at /usr/local/lib/fserver/fserver-data, then 
you should create the following symlink:
{{{
  $ sudo ln -s /usr/local/lib/fserver/fserver-data /var/www/html/fserver-data
}}}


== Starting fserver after installation ==
=== foreground mode ==
fserver can be started in foreground mode (useful for debugging), or
background mode.  To start fserver in foreground mode, 
cd to the top-level fserver directory, and run the script: ``start_server``.

The only configurable setting for fserver is the port on which HTTP operations
will take place.  

You may specify a TCP/IP port address for the server to use, on the
command line, like so:
 * $ start_server 8001

By default, port 8000 is used.

In foreground mode, the program runs directly in the terminal where
fserver was started, and log messages are displayed on the screen
as the server processes network requests.

To stop the server, use CTRL-C (possibly a few times), to interrupt
the running server.

=== background mode ===
To start fserver in background mode, use the script: ``start_local_bg_server``.
You may specify the TCP/IP port address fro the server to use, on the
command line, like so:
 * $ start_local_bg_server [<port>]

In this case, the log data from the server will be placed in the
file: /tmp/fserver-server-log.output

You can 'tail' the file to see recent log output from the server.

To stop the server, use the following command:
  $ kill $(pgrep -f fserver-server)


= Configuring Fuego to access the server =
To access the server using Fuego's ftc command, you need to configure
Fuego with the server type and the address of the server.
Put the following lines in ``fuego/fuego-ro/conf/fuego.conf``

{{{#!YellowBox
  server_type=fuego
  server_domain=localhost:8000/
}}}

The first example assumes you are running fserver on the local Fuego host,
and specified TCP/IP port 8091 for the server.

If using the global fserver service, use the following configuration:
{{{#!YellowBox
  server_type=fuego
  server_domain=fuegotest.org/cgi-bin
}}}

                     

= Accessing the server =
You can access the server using it's web interface or by sending and
receiving objects from the command line.

To access the server using a web browser, go to:
 * ht''''''tp://<''''''ip address''''''>:<''''''port''''''>/fserver.py

To access the server using the command line, use the following ftc commands:
 * ftc put-request - to put a new test request on the server
 * ftc list-requests - to list requests on the server
 * ftc run-request - to execute a request from the server
 * ftc put-run - to put run data on the server
 * ftc rm-request - to remove a request from the server
 * ftc list-runs --remote - to list runs on the server
  
== Put a request on the server ==
Use 'ftc put-request' to put a new test request on the server.  You must specify
the lab and board which will you wish to execute the test:
 * $ ftc put-request timslab:min1 Functional.zlib

== List requests on the server ==
Use 'ftc list-requests' to list the requests currently on the server.  By default,
you will only see requests that are directed to your lab (that is, the lab specified
in your fuego.conf file as your 'host_name').

If you want to see requests for all labs, or for a set of labs matching a wildcard,
specify those criteria on you command line, like so:
 * $ ftc list-requests host=franks_lab
 * $ ftc list-requests host=*

== Remove a request from the server ==
To remove a request from the server, use the 'ftc rm-request' command, and specify
the request-id, which is a string returned by the 'ftc list-requests' command.

== Running a test request from the server ==
To run a test request from the server, use the 'ftc run-request' command.

This command will default to running the next available test that is directed to
your lab (that is, that specifies a host that matches the hostname in your fuego.conf file).
 * $ ftc run-request

Note that you can specify the request-id, if there are multiple requests on the
server and you want to specify a particular one to run.
 * $ ftc run-request request-2019-06-28_20:39:58.89-timslab:bbb.json

Finally, by default ``ftc run-request`` will not push the run results back to the server.
However, you can cause this to happen using the '--put-run' argument.

{{{#!YellowBox
$ ftc run-request next --put-run
Trying to get request 'request-2019-06-28_19:47:47.19-timslab:bbb.json' from server
Executing test Functional.hello_world on board bbb (using spec default)
Running test 'Functional.hello_world' on board 'bbb' using spec 'default'
...
(test lines omitted)
...
Fuego: requested test phases complete!
Packaging run 'Functional.hello_world-default-8-bbb'
run/
run/run.json
run/devlog.txt
run/consolelog.txt
run/machine-snapshot.txt
run/build.xml
run/syslog.after.txt
run/spec.json
run/syslog.before.txt
run/prolog.sh
run/testlog.txt
Run packaged successfully, and is at: /tmp/run-Functional.hello_world-default-8-on-timslab:bbb.frp
Run package run-Functional.hello_world-default-8-on-timslab:bbb.frp was accepted by the server.
Run Functional.hello_world-default-8-bbb put to server
}}}

== Polling for test requests ==
If you want to put your host into a mode where it periodically polls the server for tests,
and executes them when they appear, you can use the script: ``fuego-core/scripts/poll_requests.sh``

This is useful if you are working with another Fuego lab, and want them
to be able to run tests on your boards.  You can leave your Fuego host
in polling mode, and execute any requests that the other lab makes that
are directed to your boards.  Poll_requests.sh will automatically put
run results back to the server, and remove each request, after it is 
executed.

== Putting runs from Fuego to fserver ==
To upload a run from Fuego to the currently configured fserver, you
first use 'ftc list-runs' to find the identifier of the run
you want to upload, and then use 'ftc put-run' to package the run
and transfer it to the server.

{{{#!YellowBox
  fuego-docker# ftc list-runs -q --where "board=bbb,test=hello_world,start_time>yesterday"
    Functional.hello_world-default-8-bbb
  fuego-docker# ftc put-run Functional.hello_world-default-8-bbb
    Packaging run 'Functional.hello_world-default-8-bbb'
run/
run/run.json
run/devlog.txt
run/consolelog.txt
run/machine-snapshot.txt
run/build.xml
run/syslog.after.txt
run/spec.json
run/syslog.before.txt
run/prolog.sh
run/testlog.txt
Run packaged successfully, and is at: /tmp/run-Functional.hello_world-default-8-on-timslab:bbb.frp
Run package run-Functional.hello_world-default-8-on-timslab:bbb.frp was accepted by the server.
Run Functional.hello_world-default-8-bbb put to server
}}}



= Issues =
There is more work needed to finish this feature.  Here is a list of items
that need to be worked on:
 * fix bug with ftc list-requests hanging sometimes (server gets stuck on response)
 * add --remote support to 'ftc list-runs'
   * add filter criteria for 'ftc list-runs --remote'
 * add support for registering a lab and board
   * ftc put-host?
   * ftc put-board?
 * add --remote support to 'ftc list-tests'
 * add 'ftc put-target-package' - for populating a build artifact cache
 * add 'ftc put-image' - for populating a build artifact cache