Release 1.6 To Do in 'raw' format
{{TableOfContents}}
Here are things to do for the 1.6 release:
= priority =
* (not done) update [[Release_1.6_Notes]] from git log
* see git log
* (not done) Add items from release notes to documentation
* decide major feature list for this release
- (not done) check which items on this list were completed
- finalize basic fserver features for this release
- (done, may not be complete) add binary test packages to fserver
- run test, get result, add to local visualization
- support board dependency cache
- add kselftest test
- fix LTP visualization bugs
- separate LTP posix test from main LTP test
- add a serial port monitor
- support provisioning
* (not done) add major features
* (not done) triage test results
* (not done) see [[Release 1.6 testing - 2023-xx-xx]]
* (not done) find blocking bugs
* (not done) fix blocking bugs
* (not done) finish [[Release 1.6 Notes]]
= blocking bugs =
* Results table bugs:
* '''docker.bc-by2.Functional.bc''' job has results table title: docker-Functional.bc-default
* '''docker.bc-by2.Functional.bc''' job has results table with links to .../Functional.bc/docker.default.6.6/testlog.txt (completely wrong)
* Jenkins errors:
* Data in older format - with error message:
* NoClassDefFoundError: hudson/matrix/MatrixAggregatable
* Dependency error with plugins
= candidate features/bugfixes list =
* finish [[Release 1.6 Notes]]
- check ftc log for ftc improvements
- check fuego-core log for fuego-core improvements
- check fuego git log for fuego improvements
* upgrade jenkins
- requires a new IPC from ftc
- probably requires setting sshkey, and setting up a default 'fuego' user
* finish features, or use existing features:
- batch tests
- convert remaining testplans to batch tests:
- testplan_agl.json
- testplan_docker.json
- testplan_fuego_*.json
- testplan_usb_filesystem.json
- testplan_mmc_filesystem
- testplan_reboot.json
- testplan_sata_filesystem.json
- testplan_sata.json
- testplan_usb_filesystem.json
- add per-board criteria files for batch_default
- eg. have bbb use fail_ok_list for GLMark, Interbench, etc.
- add batch test skiplist mechanism
- have run_test check for board skiplist, and skip test
- need TAP13 syntax for a SKIP result
- fix bug with link to sub-tests with different specs
- Fuego server (fserver)
- (done) ftc list-tests --remote
- (done) install latest fserver software on fuegotest.org
- version is 0.6.0 (git top=18c8080)
- upstream serio improvements to output flow (don't buffer output on command)
- check for discussion from March with Frank that stalled
* make new batch tests (customize these in a sensible way):
- testplan_lava.json
- testplan_ltsi.json
- testplan_lts.json
* Documentation:
- (not done) document new features
- (not done) remote test (server)
- (not done) document new tests
- try: echo "engine/tests/*" | xargs git-when-introduced.sh | sort -k 2 -V
- try: for t in tests/* ; do git-when-introduced.sh engine/$t ; done | sort -k 2 -V
= release testing =
* Do release testing
- default test plan:
- test bbb
- test docker
- test min1
- test ren1
- test rpi3-2
- test localhost
- test virtual host
- fuego test plan:
- docker
- fuego-test
- ftc-test
- bbb
- min1
- ren1
- rpi3-2
* triage test results:
- see [[Release 1.6 testing - 2019-xx-xx]]
- prioritize bugs
* fix priority bugs
= item review =
- General:
- review items on the [[Fuego_Issues_List]]
- review items on the [[Fuego_Wiki_Fixthis_List]]
- review items on [[CodeFixthisList]]
- follow release check list on [[Release Process]]
= fix bugs =
- fix bugs:
- DEFER ftc add-node help doesn't work (ftc add-nodes help does)
- make help work with singular command names
- DEFER beaglebone-serial Functional.fuego_board_check has error in pre_test
and then hangs:
{{{
Logged in as user root
echo Starting test beaglebone
ne-serial.default.Functional.fuego_board_check >> /tmp/fuego.Functional.fuego_boa
ard_check/fake-syslog' ; r=$?; echo '
ERROR: invalid literal for int() with base 10: '[ [&&serio_cmd_done&&]]'
}}}
- batch bugs:
- ftc add-jobs -b bbb -t Functional.batch_hello from outside container, with container not running, shows "Import error: jenkins module not found", when it's actually a problem accessing the container?
-
= priorities for this release =
= future planned/priorities =
- support interactive run-test (with interactive creation of board file)
== deferred this release ==
= previously deferred things=
Here is a list of previously deferred things that are still outstanding:
== deferred from 1.5 ==
- fix LTP lack of console output during test
- use nested test mechanisms:
- 'action' tests, to do: make_cache, make_report
- add parallel testing infrastructure (pipelines):
- add monitor actions
- add stress actions
- add provisioning actions
- support remote test
- test scheduling?? (no)
== deferred from 1.4 ==
- (1) full fserver support
* document how to use fserver
* how to use the existing fserver at fuegotest
* how to set up your own fserver
== deferred from 1.3 ==
- add checkbox and replot to flot plots? (maybe not)
- support fuego release self-test (probably defer to 1.4 release)
- create pre-built docker image
- more flake8 testing
- add check of parsing code to Functional.fuego_lint
- add backfire kernel module build
- parse LTP error descriptions
- process rst into html for testcases, and have it appear in Jenkins interface
- support board reboot (via BOARD_CONTROL) with another tool
- support lava, labgrid, r4d, or boruta (Pawel's) DUT control software?
- support board provisioning (via BOARD_CONTROL)
- LTS provisioning support
- remove or convert all chart_configs to proper files
- ftc set-criteria ...
- convert documentation to reStructureText
- create new user guide
- convert wiki architecture guide
- convert wiki developer reference materials