Using_Fuego_with_fserver 

Fuego wiki

Login or create account

navigation
search
toolbox
  article   edit this page   history  

Using Fuego with fserver

Contents:

Note: This is a feature of Fuego that is in prototype form. That is, this feature is currently under development, and may change as the code matures and is readied for production use. It is presented here now to allow for experimentation and feedback.

Quicklink [edit section]

The main global fserver web site is here: https://fuegotest.org/cgi-bin/fserver.py

A test fserver web site is here: https://birdcloud.org/cgi-bin/fserver.py

Introduction [edit section]

Normally, Fuego uses Jenkins for some important elements of the test automation process and lifecycle. This includes:
  • triggering of test execution
  • test scheduling (ie board scheduling and test timeout management)
  • test results storage
  • test results presentation
  • test results notifications

Some of these elements occur before a test is executed, and some occur after a test is executed. Fuego supports integration with external services for some of these actions. For example, some Fuego labs use LAVA for board provisioning and test scheduling.

This page describes 'fserver', which is a json-oriented test object manager designed to support Fuego objects.

Normally, Fuego stores test results on the test host (where Fuego is installed). These results can be queried and perused (browsed, examined, have visualizations presented) using a command line tool and via the Jenkins web interface.

However, it is also possible to send test objects, including test packages, test requests, and test results to a different service for management and processing there.

This page documents the use of Fuego with the 'fserver' test object server.

fserver design philosophy [edit section]

Fserver is current designed to be a "dumb json object store". It does not currently (as of April 2019, version 0.2.0) do any active management or processing of test objects. There are 3 different objects stored currently by fserver:
  • test objects (e.g. test definitions)
  • test requests
  • test runs (ie test results objects, or results artifacts)

It is anticipated that this list of test objects may be expanded in the future to include additional object types, including the following (potential):

  • test reviews (including ratings)
  • build artifacts (build images)
  • test target packages
  • test report definitions
  • lab definitions
  • board definitions
  • user objects (for authentication)

Fserver acts as an external repository of test objects. Note that all processing logic is managed by remote nodes that interact with the server.

For example, a user may request that a test to be executed on a board in another lab by adding a test object to the fserver respository, and also adding a test request object to the fserver repository, with the desired lab and board specified in the test request object. fserver itself does not interpret the data in the objects in any way. If the other lab (the one mentioned in the test request) is polling for jobs on fserver, that lab will note that there is a job for it to perform, and will download the job, execute it, and put a run object (test results object) back on the fserver site. Then, the first site can download the data for the test and examine or manipulate the results.

The fserver clients (the requesting host, and the Fuego host in the lab that executes the test) are completely responsible for managing the data in the fserver repository. This includes updating the status of the test request (in real time), while the job is being executed, as well as removing test requests and run results when they are no longer needed.

An fserver acts as a web-based rendezvous point for Fuego labs to communicate with each other, to coordinate distributed automated test activities.

fserver installation [edit section]

You can use fserver from an external fserver installation, or set up your own fserver server. The fserver software is pre-installed in the Fuego docker container when Fuego is installed, and no additional installation is required if you decide to use that server.

Also, there is a global instance of fserver running on the Internet on the fuegotest.org domain. You can use this fserver for experimentation if you'd like. But please do not use it for production use without first communicating with Tim Bird on the Fuego mailing list.

If you decide to install fserver directly on your host (outside the docker container) or on another machine, you may use the following instructions to manually install and start fserver.

Install the fserver server software [edit section]

To install your own instance of the fserver software (not the one included in the Fuego docker container), do the following:
  • determine an installation directory (usually /usr/local/lib/fserver, or maybe somewhere in your home directory: ~/fserver)
  • clone the fserver software from github to that directory:

The main CGI script for fserver is called fserver.py

Using fserver with an existing web server [edit section]

You can run fserver using an existing web server, or you can run fserver using a standalone local web server.

If using an existing web server, put the fserver.py script in the server's cgi-bin directory (or mark it in the web server configuration as a CGI script.)

For many Linux distros, this would mean putting the script in /usr/lib/cgi-bin.

Modify the variables in fserver.py to reflect the installation location for where you installed the server source (including the fserver-data directory).

You will also need to make sure that the web server has access to the fserver-data directory. Usually a command like this will suffice: 

  $ sudo chown -R www-data.www-data fserver-data

You will also need to provide a link so that the web server can access the fserver-data directories, so that users can download objects (such as json, yaml and package files) directly via the web server.

If your web server has its root www directory at /var/www/html, and your fserver-data directory is at /usr/local/lib/fserver/fserver-data, then you should create the following symlink: 

  $ sudo ln -s /usr/local/lib/fserver/fserver-data /var/www/html/fserver-data

Starting fserver after installation [edit section]

foreground mode [edit section]

fserver can be started in foreground mode (useful for debugging), or background mode. To start fserver in foreground mode, cd to the top-level fserver directory, and run the script: start_server.

The only configurable setting for fserver is the port on which HTTP operations will take place.

You may specify a TCP/IP port address for the server to use, on the command line, like so:

  • $ start_server 8001

By default, port 8000 is used.

In foreground mode, the program runs directly in the terminal where fserver was started, and log messages are displayed on the screen as the server processes network requests.

To stop the server, use CTRL-C (possibly a few times), to interrupt the running server.

background mode [edit section]

To start fserver in background mode, use the script: start_local_bg_server. You may specify the TCP/IP port address fro the server to use, on the command line, like so:
  • $ start_local_bg_server [<port>]

In this case, the log data from the server will be placed in the file: /tmp/fserver-server-log.output

You can 'tail' the file to see recent log output from the server.

To stop the server, use the following command: $ kill $(pgrep -f fserver-server)

Configuring Fuego to access the server [edit section]

To access the server using Fuego's ftc command, you need to configure Fuego with the server type and the address of the server. Put the following lines in fuego/fuego-ro/conf/fuego.conf 

      server_type=fuego
  server_domain=localhost:8000/

The first example assumes you are running fserver on the local Fuego host, and specified TCP/IP port 8091 for the server.

If using the global fserver service, use the following configuration: 

      server_type=fuego
  server_domain=fuegotest.org/cgi-bin

Accessing the server [edit section]

You can access the server using it's web interface or by sending and receiving objects from the command line.

To access the server using a web browser, go to:

  • http://<ip address>:<port>/fserver.py

To access the server using the command line, use the following ftc commands:

  • ftc put-request - to put a new test request on the server
  • ftc list-requests - to list requests on the server
  • ftc run-request - to execute a request from the server
  • ftc put-run - to put run data on the server
  • ftc rm-request - to remove a request from the server
  • ftc list-runs --remote - to list runs on the server

Put a request on the server [edit section]

Use 'ftc put-request' to put a new test request on the server. You must specify the lab and board which will you wish to execute the test:
  • $ ftc put-request timslab:min1 Functional.zlib

List requests on the server [edit section]

Use 'ftc list-requests' to list the requests currently on the server. By default, you will only see requests that are directed to your lab (that is, the lab specified in your fuego.conf file as your 'host_name').

If you want to see requests for all labs, or for a set of labs matching a wildcard, specify those criteria on you command line, like so:

  • $ ftc list-requests host=franks_lab
  • $ ftc list-requests host=*

Remove a request from the server [edit section]

To remove a request from the server, use the 'ftc rm-request' command, and specify the request-id, which is a string returned by the 'ftc list-requests' command.

Running a test request from the server [edit section]

To run a test request from the server, use the 'ftc run-request' command.

This command will default to running the next available test that is directed to your lab (that is, that specifies a host that matches the hostname in your fuego.conf file).

  • $ ftc run-request

Note that you can specify the request-id, if there are multiple requests on the server and you want to specify a particular one to run.

  • $ ftc run-request request-2019-06-28_20:39:58.89-timslab:bbb.json

Finally, by default ftc run-request will not push the run results back to the server. However, you can cause this to happen using the '--put-run' argument. 

    $ ftc run-request next --put-run
Trying to get request 'request-2019-06-28_19:47:47.19-timslab:bbb.json' from server
Executing test Functional.hello_world on board bbb (using spec default)
Running test 'Functional.hello_world' on board 'bbb' using spec 'default'
...
(test lines omitted)
...
Fuego: requested test phases complete!
Packaging run 'Functional.hello_world-default-8-bbb'
run/
run/run.json
run/devlog.txt
run/consolelog.txt
run/machine-snapshot.txt
run/build.xml
run/syslog.after.txt
run/spec.json
run/syslog.before.txt
run/prolog.sh
run/testlog.txt
Run packaged successfully, and is at: /tmp/run-Functional.hello_world-default-8-on-timslab:bbb.frp
Run package run-Functional.hello_world-default-8-on-timslab:bbb.frp was accepted by the server.
Run Functional.hello_world-default-8-bbb put to server

Polling for test requests [edit section]

If you want to put your host into a mode where it periodically polls the server for tests, and executes them when they appear, you can use the script: fuego-core/scripts/poll_requests.sh

This is useful if you are working with another Fuego lab, and want them to be able to run tests on your boards. You can leave your Fuego host in polling mode, and execute any requests that the other lab makes that are directed to your boards. Poll_requests.sh will automatically put run results back to the server, and remove each request, after it is executed.

Putting runs from Fuego to fserver [edit section]

To upload a run from Fuego to the currently configured fserver, you first use 'ftc list-runs' to find the identifier of the run you want to upload, and then use 'ftc put-run' to package the run and transfer it to the server. 

      fuego-docker# ftc list-runs -q --where "board=bbb,test=hello_world,start_time>yesterday"
    Functional.hello_world-default-8-bbb
  fuego-docker# ftc put-run Functional.hello_world-default-8-bbb
    Packaging run 'Functional.hello_world-default-8-bbb'
run/
run/run.json
run/devlog.txt
run/consolelog.txt
run/machine-snapshot.txt
run/build.xml
run/syslog.after.txt
run/spec.json
run/syslog.before.txt
run/prolog.sh
run/testlog.txt
Run packaged successfully, and is at: /tmp/run-Functional.hello_world-default-8-on-timslab:bbb.frp
Run package run-Functional.hello_world-default-8-on-timslab:bbb.frp was accepted by the server.
Run Functional.hello_world-default-8-bbb put to server

Issues [edit section]

There is more work needed to finish this feature. Here is a list of items that need to be worked on:
  • fix bug with ftc list-requests hanging sometimes (server gets stuck on response)
  • add --remote support to 'ftc list-runs'
    • add filter criteria for 'ftc list-runs --remote'
  • add support for registering a lab and board
    • ftc put-host?
    • ftc put-board?
  • add --remote support to 'ftc list-tests'
  • add 'ftc put-target-package' - for populating a build artifact cache
  • add 'ftc put-image' - for populating a build artifact cache

TBWiki engine 1.8.3 by Tim Bird