Jenkins >> Release_1.6_To_Do 

Fuego wiki

Login or create account

navigation
search
toolbox
  article   edit this page   history  

Release 1.6 To Do

Contents:

Here are things to do for the 1.6 release:

priority [edit section]

  • (not done) update Release_1.6_Notes from git log
    • see git log
  • (not done) Add items from release notes to documentation
  • decide major feature list for this release
    • (not done) check which items on this list were completed
    • finalize basic fserver features for this release
      • (done, may not be complete) add binary test packages to fserver
      • run test, get result, add to local visualization
    • support board dependency cache
    • add kselftest test
    • fix LTP visualization bugs
    • separate LTP posix test from main LTP test
    • add a serial port monitor
    • support provisioning
  • (not done) add major features
  • (not done) triage test results
  • (not done) find blocking bugs
  • (not done) fix blocking bugs
  • (not done) finish Release 1.6 Notes

blocking bugs [edit section]

  • Results table bugs:
    • docker.bc-by2.Functional.bc job has results table title: docker-Functional.bc-default
    • docker.bc-by2.Functional.bc job has results table with links to .../Functional.bc/docker.default.6.6/testlog.txt (completely wrong)

  • Jenkins errors:
    • Data in older format - with error message:
      • NoClassDefFoundError: hudson/matrix/MatrixAggregatable
    • Dependency error with plugins

candidate features/bugfixes list [edit section]

  • finish Release 1.6 Notes
    • check ftc log for ftc improvements
    • check fuego-core log for fuego-core improvements
    • check fuego git log for fuego improvements
  • upgrade jenkins
    • requires a new IPC from ftc
      • probably requires setting sshkey, and setting up a default 'fuego' user
  • finish features, or use existing features:
    • batch tests
      • convert remaining testplans to batch tests:
        • testplan_agl.json
        • testplan_docker.json
        • testplan_fuego_*.json
        • testplan_usb_filesystem.json
        • testplan_mmc_filesystem
        • testplan_reboot.json
        • testplan_sata_filesystem.json
        • testplan_sata.json
        • testplan_usb_filesystem.json
      • add per-board criteria files for batch_default
        • eg. have bbb use fail_ok_list for GLMark, Interbench, etc.
      • add batch test skiplist mechanism
        • have run_test check for board skiplist, and skip test
          • need TAP13 syntax for a SKIP result
      • fix bug with link to sub-tests with different specs
    • Fuego server (fserver)
      • (done) ftc list-tests --remote
        • (done) install latest fserver software on fuegotest.org
          • version is 0.6.0 (git top=18c8080)
    • upstream serio improvements to output flow (don't buffer output on command)
      • check for discussion from March with Frank that stalled
  • make new batch tests (customize these in a sensible way):
    • testplan_lava.json
    • testplan_ltsi.json
    • testplan_lts.json
  • Documentation:
    • (not done) document new features
      • (not done) remote test (server)
    • (not done) document new tests
      • try: echo "engine/tests/*" | xargs git-when-introduced.sh | sort -k 2 -V
      • try: for t in tests/* ; do git-when-introduced.sh engine/$t ; done | sort -k 2 -V

release testing [edit section]

  • Do release testing
    • default test plan:
      • test bbb
      • test docker
      • test min1
      • test ren1
      • test rpi3-2
      • test localhost
      • test virtual host
    • fuego test plan:
      • docker
      • fuego-test
      • ftc-test
      • bbb
      • min1
      • ren1
      • rpi3-2
  • triage test results:
  • fix priority bugs

item review [edit section]

fix bugs [edit section]

  • fix bugs:
    • DEFER ftc add-node help doesn't work (ftc add-nodes help does)
      • make help work with singular command names
    • DEFER beaglebone-serial Functional.fuego_board_check has error in pre_test and then hangs: 
      Logged in as user root
echo Starting test beaglebone
ne-serial.default.Functional.fuego_board_check >> /tmp/fuego.Functional.fuego_boa
ard_check/fake-syslog' ; r=$?; echo '
ERROR: invalid literal for int() with base 10: '[ [&&serio_cmd_done&&]]'

  • batch bugs:
    • ftc add-jobs -b bbb -t Functional.batch_hello from outside container, with container not running, shows "Import error: jenkins module not found", when it's actually a problem accessing the container? -

priorities for this release [edit section]

future planned/priorities [edit section]

  • support interactive run-test (with interactive creation of board file)

deferred this release [edit section]

previously deferred things [edit section]

Here is a list of previously deferred things that are still outstanding:

deferred from 1.5 [edit section]

  • fix LTP lack of console output during test
  • use nested test mechanisms:
    • 'action' tests, to do: make_cache, make_report
    • add parallel testing infrastructure (pipelines):
      • add monitor actions
      • add stress actions
    • add provisioning actions
  • support remote test
    • test scheduling?? (no)

deferred from 1.4 [edit section]

  • (1) full fserver support
    • document how to use fserver
    • how to use the existing fserver at fuegotest
    • how to set up your own fserver

deferred from 1.3 [edit section]

  • add checkbox and replot to flot plots? (maybe not)
  • support fuego release self-test (probably defer to 1.4 release)
  • create pre-built docker image
  • more flake8 testing
    • add check of parsing code to Functional.fuego_lint
  • add backfire kernel module build
  • parse LTP error descriptions
  • process rst into html for testcases, and have it appear in Jenkins interface
  • support board reboot (via BOARD_CONTROL) with another tool
    • support lava, labgrid, r4d, or boruta (Pawel's) DUT control software?
  • support board provisioning (via BOARD_CONTROL)
    • LTS provisioning support
  • remove or convert all chart_configs to proper files
  • ftc set-criteria ...
  • convert documentation to reStructureText
    • create new user guide
    • convert wiki architecture guide
    • convert wiki developer reference materials

TBWiki engine 1.8.3 by Tim Bird