Release 1.6 testing - 2024-03-26 in 'raw' format
{{TableOfContents}} Here are some notes on testing the 1.6 release, using the next branch (with new docker Debian version (bullseye) and Jenkins version (2.414.1) The container is: fuego-322-container, corresponding to directory: /home/tbird/work/fuego/next-2024-03-22/fuego See an overview issues table at [[Release test results]] = to do = - categorize all bugs - likely location: core, test, board, toolchain, lab, unknown - severity: high, medium, low - time to fix: short, long - priority: 5=high, 1=low, 0=defer Resolution key: - CANDIDATE - is a candidate for fixing this release - RESEARCH - needs more research to categorize and prioritize - TRANSIENT - problem went away with subsequent test and no change in Fuego - LOCAL (lab,board) - problem was with setup, config, board status, board, or toolchain, and issue can be ignored - IGNORE - ignore (not a problem) - FIXED - problem fixed with change in Fuego - DEFER - problem exists, but will not be fixed in this release = uncategorized issues list = == deferred from 1.4 release == See http://fuegotest.org/wiki/Release_1.5_testing_-_2019-05-02#deferred_from_1.4_release == new errors == - xx. (distro) '''docker.default.Functional.fuego_board_check''' - missing /sbin/reboot - /sbin/reboot is present on host as a symlink to /bin/systemctl - this is expected - xx. Can't easily navigate to LTP individual test logs in Jenkins interface = old errors = - [[Benchmark.GLMark]] - build failure on multiple platforms (missing glew.h, missing libSDL) = Errors by test = - Whetstone - (run phase, bbb & min1) libm.so.6: version `GLIBC_2.29' not found - probable toolchain problem? (toolchain too new for board)? - min1.default.Functional.netperf - build error 0 Dhrystone - (run phase, rpi3-2) bad intepreter (error is 'dhrystone not found'), but the file is present. Interpreter is /lib/ld-linux-armhf.so.3, but rpi3-2 has ld-linux /lib/ld-linux-aarch64.so.1 - did I upgrade the distro on rpi3-2 to 64-bit? low priority: - Dhrystone - (processing, docker & ren1) not enough loops (using spec '500M' works) = Errors by board = == bbb board == # ftc list-runs --where "start_time>4 days ago,status!=PASS,board=bbb" {{{ .. add data here ... }}} == min1 board == Here are the tests that had issues: - min1.default.Benchmark.Whetstone - libm.so.6 version GLIBC_2.29 not found {{{ # ftc list-runs --where board=min1,status!=PASS ... add data here... }}} ---- Here are the LTP non-PASS results: {{{ # ftc gen-report --where board=min1,test=LTP,tguid:result!=PASS ... add data here ... }}} --- Here is the reason for each LTP failure (with some comments) {{{ UPDATE this table!! ------------------------------------ tguid tguid:result ------------------------------------ syscalls.fanotify06 FAIL - segfault syscalls.msgctl11 FAIL - queue mismatch syscalls.open10 FAIL - setgid not set syscalls.creat08 FAIL - setgid not set sem_init.sem_init_7-1 SKIP - ??? (RESEARCH) ------------------------------------ }}} == ren1 board == # ftc list-runs --where board=ren1,status!=PASS {{{ ..add data here .. }}} ---- Here is the LTP report for non-PASS tests: # ftc gen-report --where board=ren1,test=LTP,tguid:result!=PASS {{{ .. add data here .. }}} ---- Here are LTP details: To get sorted testcase names: * $ ftc gen-report --where board=ren1,test=LTP,tguid:result!=PASS --fields tguid,tguid:result | sort {{{ .. add data here .. }}} == fuego-test board == UPDATE THIS DATA - (CANDIDATE) '''fuego-test.default.Functional.fuego_abort''' - reports SKIP, but should be OK (behavior is correct) - should turn this test into a nested test, that checks for correct messages in the console log - (CANDIDATE) '''fuego-test.default.fuego_board_check''' - missing /sbin/reboot in container - '''fuego-test.default.Functional.fuego_ftc_test''' - testcase 6 (ftc query-board differences) - '''fuego-test.default.Functional.fuego_lint''' - expected failures - '''fuego-test.default.Functional.fuego_release_test''' - - '''fuego-test.default.Functional.fuego_slow_log''' - # ftc list-runs --where board=fuego-test,status!=PASS {{{ UPDATE THIS DATA Runs on this host: Test Status --------------------------- ------ Functional.fuego_abort SKIP - OK (CANDIDATE) Functional.fuego_board_check FAIL - missing /sbin/reboot Functional.fuego_compare_reports FAIL - missing baseline Functional.fuego_ftc_test FAIL - (FIXED) testcase 3 - ftc help, testcase 6 - quoting of variables Functional.fuego_lint FAIL - 559 errors in ftc (ok for now) Functional.fuego_release_test FAIL - (CANDIDATE) password is required Functional.fuego_slow_log FAIL - (FIXED) syntax error near unexpected token done }}}