Using Fuego with fserver
Note: This is a feature of Fuego that is in prototype form. That is, this feature is currently under development, and may change as the code matures and is readied for production use. It is presented here now to allow for experimentation and feedback.
A test fserver web site is here: https://birdcloud.org/cgi-bin/fserver.py
- triggering of test execution
- test scheduling (ie board scheduling and test timeout management)
- test results storage
- test results presentation
- test results notifications
Some of these elements occur before a test is executed, and some occur after a test is executed. Fuego supports integration with external services for some of these actions. For example, some Fuego labs use LAVA for board provisioning and test scheduling.
This page describes 'fserver', which is a json-oriented test object manager designed to support Fuego objects.
Normally, Fuego stores test results on the test host (where Fuego is installed). These results can be queried and perused (browsed, examined, have visualizations presented) using a command line tool and via the Jenkins web interface.
However, it is also possible to send test objects, including test packages, test requests, and test results to a different service for management and processing there.
This page documents the use of Fuego with the 'fserver' test object server.
- test objects (e.g. test definitions)
- test requests
- test runs (ie test results objects, or results artifacts)
It is anticipated that this list of test objects may be expanded in the future to include additional object types, including the following (potential):
- test reviews (including ratings)
- build artifacts (build images)
- test target packages
- test report definitions
- lab definitions
- board definitions
- user objects (for authentication)
Fserver acts as an external repository of test objects. Note that all processing logic is managed by remote nodes that interact with the server.
For example, a user may request that a test be executed on a board in another lab by adding a test object to the fserver respository, and also adding a test request object to the fserver repository, with the desired lab and board specified in the test request object. fserver itself does not interpret the data in the objects in any way. If the other lab (the one mentioned in the test request) is polling for jobs on fserver, that lab will note that there is a job for it to perform, and will download the job, execute it, and put a run object (test results object) back on the fserver site. Then, the first site can download the data for the test and examine or manipulate the results.
The fserver clients (the requesting host, and the Fuego host in the lab that executes the test) are completely responsible for managing the data in the fserver repository. This includes updating the status of the test request (in real time), while the job is being executed, as well as removing test requests and run results when they are no longer needed.
An fserver acts as a web-based rendezvous point for Fuego labs to communicate with each other, to coordinate distributed automated test activities.
Also, there is a global instance of fserver running on the Internet on
fuegotest.org domain. You can use this fserver for experimentation
if you'd like. But please do not use it for production use without
first communicating with Tim Bird on the Fuego mailing list.
If you decide to install fserver directly on your host (outside the docker container) or on another machine, you may use the following instructions to manually install and start fserver.
- determine an installation directory (usually /usr/local/lib/fserver, or maybe somewhere in your home directory: ~/fserver)
- clone the fserver software from github to that directory:
- $ git clone https://github.com/tbird20d/fserver.git ~/fserver
The main CGI script for fserver is called fserver.py
If using an existing web server, put the fserver.py script in the server's cgi-bin directory (or mark it in the web server configuration as a CGI script.)
For many Linux distros, this would mean putting the script in /usr/lib/cgi-bin.
Modify the variables in fserver.py to reflect the installation location for where you installed the server source (including the fserver-data directory).
You will also need to make sure that the web server has access to the fserver-data directory. Usually a command like this will suffice:
$ sudo chown -R www-data.www-data fserver-data
You will also need to provide a link so that the web server can access the fserver-data directories, so that users can download objects (such as json, yaml and package files) directly via the web server.
If your web server has its root www directory at /var/www/html, and your fserver-data directory is at /usr/local/lib/fserver/fserver-data, then you should create the following symlink:
$ sudo ln -s /usr/local/lib/fserver/fserver-data /var/www/html/fserver-data
The only configurable setting for fserver is the port on which HTTP operations will take place.
You may specify a TCP/IP port address for the server to use, on the command line, like so:
- $ start_server 8001
By default, port 8000 is used.
In foreground mode, the program runs directly in the terminal where fserver was started, and log messages are displayed on the screen as the server processes network requests.
To stop the server, use CTRL-C (possibly a few times), to interrupt the running server.
start_local_bg_server. You may specify the TCP/IP port address fro the server to use, on the command line, like so:
- $ start_local_bg_server [<port>]
In this case, the log data from the server will be placed in the file: /tmp/fserver-server-log.output
You can 'tail' the file to see recent log output from the server.
To stop the server, use the following command: $ kill $(pgrep -f fserver-server)
The first example assumes you are running fserver on the local Fuego host, and specified TCP/IP port 8091 for the server.
If using the global fserver service, use the following configuration:
To access the server using a web browser, go to:
- http://<ip address>:<port>/fserver.py
To access the server using the command line, use the following ftc commands:
- ftc put-request - to put a new test request on the server
- ftc list-requests - to list requests on the server
- ftc run-request - to execute a request from the server
- ftc put-run - to put run data on the server
- ftc rm-request - to remove a request from the server
- ftc list-runs --remote - to list runs on the server
- $ ftc put-request timslab:min1 Functional.zlib
If you want to see requests for all labs, or for a set of labs matching a wildcard, specify those criteria on you command line, like so:
- $ ftc list-requests host=franks_lab
- $ ftc list-requests host=*
This command will default to running the next available test that is directed to your lab (that is, that specifies a host that matches the hostname in your fuego.conf file).
- $ ftc run-request
Note that you can specify the request-id, if there are multiple requests on the server and you want to specify a particular one to run.
- $ ftc run-request request-2019-06-28_20:39:58.89-timslab:bbb.json
Finally, by default
ftc run-request will not push the run results back to the server.
However, you can cause this to happen using the '--put-run' argument.
This is useful if you are working with another Fuego lab, and want them to be able to run tests on your boards. You can leave your Fuego host in polling mode, and execute any requests that the other lab makes that are directed to your boards. Poll_requests.sh will automatically put run results back to the server, and remove each request, after it is executed.
- fix bug with ftc list-requests hanging sometimes (server gets stuck on response)
- add --remote support to 'ftc list-runs'
- add filter criteria for 'ftc list-runs --remote'
- add support for registering a lab and board
- ftc put-host?
- ftc put-board?
- add --remote support to 'ftc list-tests'
- add 'ftc put-target-package' - for populating a build artifact cache
- add 'ftc put-image' - for populating a build artifact cache