{{TableOfContents}}

Here are some miscelaneous notes on the Jenkins integration refactoring done
by Daniel Sangorrin:

= basic installation =
To use the new system do:
Quickstart:
 * $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
 * $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
 * $ cd fuego/
 * $ ./install.sh
 * $ ./fuego-host-scripts/docker-create-container.sh
 * $ ./fuego-host-scripts/docker-start-container.sh
 * docker# fuego-create-node --board docker
 * docker# fuego-create-jobs --board docker --testplan testplan_docker --distrib nosyslogd.dist

Then, start the "docker.testplan_docker.batch" job which will trigger the rest of jobs.

== Tim's customizations ==
Install my test board:
 * $ cp ~/work/fuego/fuego/userdata/conf/boards/bbb-poky-sdk.board ~/work/fuego-daniel/fuego/fuego-ro/conf/boards
 * docker# fuego-create-node --board bbb-poky-sdk
 * docker# fuego-create-jobs --board bbb-poky-sdk --testplan testplan_docker --distrib nosyslogd.dist

Install my toolchain:
 * $ sudo su
 * $ mkdir -p /fuego-ro/toolchains
 * $ cd /fuego-ro/toolchains
 * $ cp ~/work/yocto/build-quemuarm/yocto/poky/build-qemuarm/tmp/deploy/sdk/*.sh .
 * $ ./*snapshot.sh
   * enter target directory for SDK: /fuego-ro/toolchains/poky
 * $ cd ~/work/fuego-daniel/fuego/fuego-ro/toolchains
 * $ cp -a /fuego-ro/toolchains/poky .
 * $ cp ~/work/fuego/fuego/userdata/toolchains/poky-qemuarm-tools.sh .
 * $ edit poky-qemuarm-tools.sh to use correct path

this did not work
 * $ ln -s /home/1000072457/work/fuego-daniel/fuego/fuego-ro /fuego-ro
 * cd /fuego-ro/toolchains
 * ./*snapshot.sh
 * remove all occurences of '/home/1000072457/work/fuego-daniel/fuego' from environment-setup-armv53-poky-linux-gnueabi

this did not work
 * docker# cd /fuego-ro/toolchains
 * docker# ./*snapshot.sh
   * enter target directory for SDK: /fuego-ro/toolchains/poky
     * I get the message cannot mkdir: Read-only file system




= Features/Changes =
Here is a list of the most fundamental modifications that were added:
 - automatized the version upgrade process. So far I have updated Jenkins 3 times quite smoothly.
   - The current version is the latest LTS version (2.31.1 LTS)
 - reduced plugin dependencies to the minimum (only 2, and they can be made optional).
 - removed dependencies on all groovy scripts.
 - Userdata is gone. Instead we now have fuego-ro and fuego-rw
    - These and fuego-core are mounted as external docker volumes.
    - Only fuego-rw, which contains logs and build folders, is mounted in RW mode.
    - also fixed the uid/gid of jenkins so that it matches the one in the host.
    - This means you can develop comfortably on your host, with GUI tools, and don't fear a buggy script deleting your folders.
 - added a  tool for generating jobs from testplans.

Creating jobs from testplans is powerful. It allows you, for example, to easily specify the timeout for each test on your board.
The tool also generates the corresponding batch job. For example, you can easily trigger all 
tests for your board and a specific testplan periodically.

[Note] This will also enable the creation of scripts for comparing test results across
different boards and test parameters (on my TODO list).

Other less fundamental changes include:
 - I added support for using the docker container itself as a target board (e.g.: for a quickstart or unit tests)
 - I fixed the flot plugin (java and javascript fixes)
 - Now you can click on a node and see which jobs (and testplans) are assigned to your board. This was one of my old feature requests and makes Fuego's GUI easier to use. 
 - I added quite a few fixes and improvements to the core engine scripts and tests (too detailed to describe here).
 - I added Excel output support for IOzone
 - Faster docker build time (ARMhf compiler installation is now optional, no latex..)
 - I put a fixed name to the fuego container, instead of the "last_container_id" file.
 - I removed the inotify script
 - I added scripts for easily removing the docker container/image.




= problem installing toolchain =
I couldn't install the toolchain into /fuego-ro/toolchains, because
/fuego-ro is mounted read-only.  I had to install the toolchain to
/fuego-ro/toolchains on my host, then as root copy the directory to

/home/<user>/work/fuego/fuego/fuego-ro/toolchains - where it then
appeared in the docker container in /fuego-ro/toolchains.

You have to use the same installation path on the host as it will
be in the container, because the environment setup script hardcodes
the path when the SDK is installed.


= can't create image due to debian server network problem =
I had a problem completing the build of the docker image, due to an
intermittent problem accessing debian servers during the build.

Here is an error I saw:
{{{#!YellowBox
Get:95 http://httpredir.debian.org/debian/ jessie/main libatk-wrapper-java all 0.30.5-1 [30.3 kB]
Err http://httpredir.debian.org/debian/ jessie/main libavahi-glib1 amd64 0.6.31-5
  Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
}}}

Daniel says:
{{{#!YellowBox
Yes, I had the same problem yesterday when I installed it on a different machine.
It seems that httpredir.debian.org is not as reliable as using a local mirror directly.
I'm not sure if I should add retries or put a default debian server as it used to be.
Could you change httpredir by the US mirror (http://ftp.us.debian.org/debian) and try again?
}}}

= Tim's notes from March 7 comparison =
== differences ==
 * frontend-install only has:
   * flot
   * fuego-install-plugins.sh, which installs:
     * description-setter
     * pegdown-formatter
 * userdata moved to:
   * fuego-ro, which has:
     * conf/boards/docker.board, lager.board, qemu-arm.board, etc.
     * scripts/nodes/fuego-create-node
     * scripts/nodes/fuego-delete-node
     * scripts/jobs/fuego-create-jobs
     * scripts/jobs/fuego-delete-jobs
     * toolchains/tools.sh, poky-qemuarm-tools.sh
   * fuego-rw, which has:
     * buildzone
     * logs
     * work

== things to fix ==
 * fuego-core/engine/scripts/ftc is missing commit to add debug option
   * I forgot to push this after the Portland trip
   * It's small and disposable as a patch (that is, it's not important to integrate it)
 * fuego/Dockerfile is missing ttc installation
   * but fuego-core/engine/overlays/base/base-board.fuegoclass has ttc support in the ov_* functions
 * post_test is not run
   * no test directories were removed on target
 * LTP did not have enough space to run on the device
   * my bbb only has about 300 meg. free


== issues ==
 * there's no documentation on providing your own image or container names
   * install.sh [<my_image_name>]
   * fuego-host-scripts/docker-create-container.sh [<my_image_name> [<my_container_name>]]
   * fuego-host-scripts/docker-start-container.sh [<my_container_name>]
 * it would be nice to put fuego-core, fuego-ro and fuego-rw under /opt/fuego.
   * for toolchain installation, it would be handy to have toolchains with the same path on the host as in the container:
      * /opt/fuego/fuego-ro/toolchains in both the container and on the host
 * The build shell command for jobs could (and should) be simplified:
   * TESTNAME should always be fuego_test.sh
     * this requires renaming all base scripts in all tests
   * DISTRIB should be in the board file
   * TESTPLAN should be able to be parsed from JOB_NAME
   * is NODE_NAME used for the board?
   * just call "ftc run-test"
     * have it detect TESTNAME, DISTRIB, TESTPLAN, NODE_NAME
     * have it detect it is being run from a Jenkins job (check for JENKINS_URL?)
 * tests directories are placed at the root on the target:
   * /fuego.Benchmark.dbench (instead of /home/a/fuego.Benchmark.dbench)
   * new system uses BOARD_TESTDIR instead of FUEGO_HOME




== questions ==
 * question: does Dockerfile use ../fuego-core, or clone it's own?
   * answer: docker-create-container.sh uses ../../fuego-core