Test tasks
Administering the system [edit section]
- install the system
- git clone..., install.sh, docker_create_container.sh, docker_start_container.sh
- add a board
- write board file, select distro, add node in Jenkins
- add a test
- write test script, write spec, write plan, write parser.py, download tarfile, add job in Jenkins
- add a toolchain
- install toolchain in container,
- add a user
- use Jenkins interface
- install a test
- (no current method for a new test)
- modify a board
- edit Jenkins target node and/or edit board file
- or, use 'ftc set' command
- modify a test
- edit test materials, write patches, edit specs, edit job in Jenkins
- modify a user
- use Jenkins interface
- delete a board
- delete a test
- create a test plan
- create a test spec
Executing tests [edit section]
- manually run a single test on a single board
- Use jenkins interface, select test, select "Run test now", select board, select testplan, set options in Jenkins dialog, select 'run test'
- or, find test name with ftc, find board name with ftc, do 'ftc run <board> <test> <options>
- configure a trigger to automatically run a single test on a single board
- use Jenkins to create a new job which triggers the job
- run multiple tests on a single board
- run a single test on multiple boards
- run a test from the Jenkins interface
- run a test from the command line
Viewing results [edit section]
- see summary of results from last few tests run on all boards
- look at History tab in Jenkins
- or, select "Test Run History" in left-side menu
- see summary of results for a single board
- click on target node, and view "Tests executed on target"
- click on target status, then target node, then view "Tests executed on target"
- see summary of results for a single test
- click on test, and look at left-side pane ("Test run history")
Collaborating [edit section]
- access shared tests
- access shared board configurations
- access shared test results
- share a test
- share a board configuration
- share test results
division of labor [edit section]
This section describes what parts of the system are in charge of or are related to varies aspects of the tasks described above:- Jenkins - provides user interface
- Jenkins - provides for test triggers
- script system (specs) - provides for variations on a single test
- script system (plans) - provides for selection of multiple parameters for a test???
- script system (overlays) - allows for inheritance of board and distro variables and functions
- batch system
- report generation system
- plotting
Notes [edit section]
- ftc does not currently support scheduling of jobs. It just executes the test on the target immediately.