{{TableOfContents}}

Here are things to do for the 1.5 release:

= priority =
 * fix blocking bugs
 * (done) finish [[Release 1.5 Notes]]
 * triage test results
   * (done) see [[Release 1.5 testing - 2019-05-02]]
   * (done) see [[Release 1.5 testing - 2019-08-13]]
 * copy files from fuego-1.5 wiki that are different


= blocking bugs =
 * Results table bugs:
   * '''docker.bc-by2.Functional.bc''' job has results table title: docker-Functional.bc-default
   * '''docker.bc-by2.Functional.bc''' job has results table with links to .../Functional.bc/docker.default.6.6/testlog.txt (completely wrong)

   * (FIXED) '''min1.default.Functional.LTP''' does not show min1-Functional.LTP results table (but it shows ren1 results table)
     * ltp_process.py not getting called, due to ORIG_PATH handling

 * Jenkins errors:
   * Data in older format - with error message:
     * NoClassDefFoundError: hudson/matrix/MatrixAggregatable
   * Dependency error with plugins


= general list =
 * (1) finish [[Release 1.5 Notes]]
 * finish features:
   - batch tests
     - convert remaining testplans to batch tests:
       - testplan_agl.json
       - (done) testplan_bc_*.json
       - (done) testplan_default.json
       - testplan_docker.json
       - testplan_fuego_*.json
       - (done) testplan_hello_world_*.json
       - testplan_lava.json
       - testplan_ltsi.json
       - testplan_lts.json
       - testplan_mmc_filesystem.json
       - testplan_reboot.json
       - testplan_sata_filesystem.json
       - testplan_sata.json
       - (done) testplan_smoketest.json
       - testplan_usb_filesystem.json
     - add per-board criteria files for batch_default
        - eg. have bbb use fail_ok_list for GLMark, Interbench, etc.
     - add batch test skiplist mechanism
        - have run_test check for board skiplist, and skip test
          - need TAP13 syntax for a SKIP result
     - fix bug with link to sub-tests with different specs
   - (done) squad backend prototype
   - (done) user-configurable Jenkins port
   - Fuego server (fserver)
     - ftc list-tests --remote
       - install latest fserver software on fuegotest.org
   - serio improvements to output flow (don't buffer output on command)
     - check for discussion from March with Frank that stalled
 * Documentation:
   - document batch tests
     - (done) document [[function_run_test|run_test]]
     - (done) document [[function_allocate_next_batch_id|allocate_next_batch_id]]
     - (done) creating jobs for batch tests
     - links in the batch results to job pages
     - generating reports for batch test results
   - document ftc put-run squad
     - configuration for squad server
   - document user configurable port
   - document default port  8090
   - document new tests
      - try: echo "engine/tests/*" | xargs git-when-introduced.sh | sort -k 2 -V
      - try: for t in tests/* ; do git-when-introduced.sh engine/$t ; done | sort -k 2 -V

      - Functional.linaro
{{{
tests/Functional.fuego_batch1/fuego_test.sh: v1.4.0-21-g3dc481c
tests/Functional.linaro/fuego_test.sh: v1.4.0-36-g6f5b706
tests/Functional.nscd/fuego_test.sh: v1.4.0-41-g6781800
tests/Functional.module_init_tools/fuego_test.sh: v1.4.0-43-g38cfbfb
tests/Functional.batch_default/fuego_test.sh: v1.4.0-62-gfbfdfff
tests/Functional.ipmi/fuego_test.sh: v1.4.0-65-gcee2f9c
tests/Functional.openhpid/fuego_test.sh: v1.4.0-66-g2ab8b5c
tests/Functional.openct/fuego_test.sh: v1.4.0-67-ga2ff574
tests/Functional.brctl/fuego_test.sh: v1.4.0-68-gbd32eaa
tests/Functional.multipathd/fuego_test.sh: v1.4.0-69-g8d1cf2e
tests/Functional.ptest/fuego_test.sh: v1.4.0-71-g4cc8d3f
tests/Functional.libxml/fuego_test.sh: v1.4.0-82-g019fc83
tests/Functional.batch_bc/fuego_test.sh: v1.4.0-88-gbbfa30c
tests/Functional.iperf3_server/fuego_test.sh: v1.4.0-89-g2dd180a
tests/Functional.batch_smoketest/fuego_test.sh: v1.4.0-93-g82e2474
tests/Functional.batch_hello/fuego_test.sh: v1.4.0-98-gab4a9ec

}}}
   - remote test (server)


= release testing =
 * Do release testing
   - default test plan:
     - test bbb
     - test docker
     - test min1
     - test ren1
     - test rpi3-2
     - test localhost
     - test virtual host
   - fuego test plan:
     - docker
     - fuego-test
     - ftc-test
     - bbb
     - min1
     - ren1
     - rpi3-2
 * triage test results:
   - see [[Release 1.5 testing - 2019-05-02]]
   - prioritize bugs
 * fix priority bugs

= item review =
 - General:
   - review items on the [[Fuego_Issues_List]]
   - review items on the [[Fuego_Wiki_Fixthis_List]]
   - review items on [[CodeFixthisList]]
   - follow release check list on [[Release Process]]
 

= fix bugs =
 - fix bugs:
   - DEFER ftc add-node help doesn't work (ftc add-nodes help does)
     - make help work with singular command names
   - DEFER beaglebone-serial Functional.fuego_board_check has error in pre_test
     and then hangs:
{{{
Logged in as user root
echo Starting test beaglebone
ne-serial.default.Functional.fuego_board_check >> /tmp/fuego.Functional.fuego_boa
ard_check/fake-syslog' ; r=$?; echo '
ERROR: invalid literal for int() with base 10: '[ [&&serio_cmd_done&&]]'
}}}

   - batch bugs:
     - ftc add-jobs -b bbb -t Functional.batch_hello from outside container, with container not running, shows "Import error: jenkins module not found", when it's actually a problem accessing the container?
     -

 - things to commit



= priorities for this release =

= future planned/priorities =
 - support squad back-end
 - support interactive run-test (with interactive creation of board file)

== deferred this release ==
 - depend on nested test mechanisms:
   - 'action' tests, to do: make_cache, make_report
   - monitor tests (needs parallel testing infrastructure)
   - stress tests (needs parallel testing infrastructure)
 - support remote test
   - test scheduling?? (no)
 - (1) full fserver support
   * document how to use fserver
   * how to use the existing fserver at fuegotest
   * how to set up your own fserver


= things done =

= previously deferred things=
Here is a list of previously deferred things that are still outstanding:

== deferred from 1.3 ==
 - add checkbox and replot to flot plots? (maybe not)
 - support fuego release self-test (probably defer to 1.4 release)
 - create pre-built docker image
 - more flake8 testing
   - add check of parsing code to  Functional.fuego_lint
 - add backfire kernel module build
 - parse LTP error descriptions
 - process rst into html for testcases, and have it appear in Jenkins interface
 - support board reboot (via BOARD_CONTROL) with another tool
   - support lava, labgrid, r4d, or boruta (Pawel's) DUT control software?
 - support board provisioning (via BOARD_CONTROL)
   - LTS provisioning support
 - remove or convert all chart_configs to proper files
 - ftc set-criteria ...
 - convert documentation to reStructureText
   - create new user guide
   - convert wiki architecture guide
   - convert wiki developer reference materials