Here are some notes from the 2017 Fuego BOF

materials [edit section]

= materials =
 * The slides are here: http://elinux.org/images/f/f6/Fuego-Status-and-Roadmap-ELC-2017-02.pdf
 * The video for the BOF is available here:
https://youtu.be/DL1KJuVcbcY

Tim's notes [edit section]

= Tim's notes =
 * need to make tests soon
 * see slides pages 44-45 for notes
 * transport: ADB support - Run adbd outside container (on host)
   * container doesn’t have to know about usb changes
 * could use transport=local for host as DUT
   * Now currently used for docker container as DUT
 * Bypassing the build step
   * would make it so end-user doesn't have to build
     * may allow for use of Fuego without docker container or Jenkins (java)
   * It’s OK to have something as a build cache, but make sure not to lose ability to build from source
   * Don’t allow “magic binaries” that someone can’t rebuild
 * Bisect
   * Should be a tool outside Fuego to bisect basedon Fuego test result
   * Ftc needs to return proper error code
   * Maybe provide an example for how to do it
 * Image Deploy, re-flash
   * Since LAVA does these, and AGL already uses LAVA, these are not high priority at the moment

• demo: * Tim demoed: * ftc list-tests * ftc package-test * ftc put-test * show tests, requests, runs on server * show test detail * show run detail * whole run is not untarred on server - can't just click on a log to see it * ftc put-request * see bug not below * ftc list-requests * show runs for timdesk only (use wildcard "tim*") * ftc put-request - server did not create the wrapper for the json file

 * demo:
   * Tim demoed:
     * ftc list-tests
     * ftc package-test
     * ftc put-test
     * show tests, requests, runs on server
       * show test detail
       * show run detail
          * whole run is not untarred on server - can't just click on a log to see it
     * ftc put-request
       * see bug not below
     * ftc list-requests
       * show runs for timdesk only (use wildcard "tim*")
   * ftc put-request - server did not create the wrapper for the json file

transcript notes [edit section]

= transcript notes =
Here are some questions or responses from the audience not caught on audio
(Tim didn't repeat the question or comment for the video. tsk tsk.)

• at 6:05 - Is AGL using dynamic parameters? Are tests launched manually or triggered by some automatic thing? No. Tests are triggered. * at 6:27 - same question to Toshiba? Daniel responded that it's easy to change the Jenkins job configuration for the test, to alter the parameter manually, before you do the "build" (launch the test). * dynamic parameters appears to be a low priority * at 13:20 - comment about running adbd outside the docker container, so it can see the usb changes on the host from the target disconnecting and reconnecting * start adbd before the container runs, and adb inside the container to connect to it (over network or pipe?) * at 14:02 - Tim leaves for a bit to get his notebook to write discussions in * at 15:15 - Tim asked if LAVA integration was at the transport layer. Jan-Simon said he used the hook in pretest * at 15:52 - Daniel mentioned the "local" transport, which is used to run a test on the local machine (inside the Docker container) * this might be useful for running tests on the host * at 16:24 - Tim asked if the local transport put the test in docker or on the host. The answer was "in docker". * at 16:40 - Daniel asked if you could avoid running Jenkins in this case. Tim said yes, it should be possible. * at 16:55 - Someone asked if Fuego handles the bootloader layer, flashing the image and intelligently booting. Tim said no. * at 26:09 - comment about doing builds separate from the actual test run, so that the Fuego system doesn't hold the board reserved and unavailable during the build. This appears to be the cause of a lot of LAVA timeouts. Some builds can take a long time, and the build time is being included in the total board reservation time. * at 27:15 - comment about it being important to not have "magic" binaries that people can't rebuild * at 27:53 - comment that building packages is good for testing the SDK for a board. * at 28:40 - comment (didn't catch it), but something about running jobs on kernelCI lab boards * at 30:18 - Daniel clarifies the meaning of "overrides: the pretest should be able to automatically select" * I believe he means that fuego should automatically detect some settings, particularly the distro settings, so the user doesn't have to do this. * at 35:35 - (Daniel - unknown comment) * at 36:20 - (Daniel - unknown comment - related to using Jenkins labels for parallel testing on multiple devices) * at 39:20 - Daniel comment on bisect - it's valuable to do and is very handy to find stuff * at 40:07 - Olaf comment that bisect should be outside the framework * at 40:45 - (I only got to minute 40)

 * at 6:05 - Is AGL using dynamic parameters?  Are tests launched manually or triggered by some automatic thing?  No. Tests are triggered.
 * at 6:27 - same question to Toshiba? Daniel responded that it's easy to change the Jenkins job configuration for the test, to alter the parameter manually, before you do the "build" (launch the test).
   * dynamic parameters appears to be a low priority
 * at 13:20 - comment about running adbd outside the docker container, so it can see the usb changes on the host from the target disconnecting and reconnecting
   * start adbd before the container runs, and adb inside the container to connect to it (over network or pipe?)
 * at 14:02 - Tim leaves for a bit to get his notebook to write discussions in
 * at 15:15 - Tim asked if LAVA integration was at the transport layer.  Jan-Simon said he used the hook in pretest
 * at 15:52 - Daniel mentioned the "local" transport, which is used to run a test on the local machine (inside the Docker container)
   * this might be useful for running tests on the host
 * at 16:24 - Tim asked if the local transport put the test in docker or on the host.  The answer was "in docker".
 * at 16:40 - Daniel asked if you could avoid running Jenkins in this case.  Tim said yes, it should be possible.
 * at 16:55 - Someone asked if Fuego handles the bootloader layer, flashing the image and intelligently booting.  Tim said no.
 * at 26:09 - comment about doing builds separate from the actual test run, so that the Fuego system doesn't hold the board reserved and unavailable during the build.  This appears to be the cause of a lot of LAVA timeouts.  Some builds can take a long time, and the build time is being included in the total
board reservation time.
 * at 27:15 - comment about it being important to not have "magic" binaries that people can't rebuild
 * at 27:53 - comment that building packages is good for testing the SDK for a board.
 * at 28:40 - comment (didn't catch it), but something about running jobs on kernelCI lab boards
 * at 30:18 - Daniel clarifies the meaning of "overrides: the pretest should be able to automatically select"
   * I believe he means that fuego should automatically detect some settings, particularly the distro settings, so the user doesn't have to do this.
 * at 35:35 - (Daniel - unknown comment)
 * at 36:20 - (Daniel - unknown comment - related to using Jenkins labels for parallel testing on multiple devices)
 * at 39:20 - Daniel comment on bisect - it's valuable to do and is very handy to find stuff
 * at 40:07 - Olaf comment that bisect should be outside the framework
 * at 40:45 - 
(I only got to minute 40)