release-1.2-test-2017-09-27 Notes
flat problem list [edit section]
Resolution key:- FIXED - problem fixed with change in Fuego
- TRANSIENT - problem went away with subsequent test and no change in Fuego
- LAB ERROR - problem was with setup, config, or board status, issue can be ignored
Bugs seen:
- bbb.default.Benchmark.Interbench - aborted due to timeout of 40 minutes
- hangs in simulated CPU audio test - calloc cannot allocate memory
- leaves 10 zombies on the target
- bbb.default.Benchmark.Interbench - had Rebuild set to 'true' after job creation
- bbb - lots of tests failed because bbb wedged after a test
- (should look at history and see which test ran last on 9/27)
- bbb.default.Benchmark.blobsallad - build error - missing SDL.h
- bbb.default.Benchmark.GLMark - build error - missing SDL.h
- bbb.default.Benchmark.gtkperf - build error - configure error checking for gtk+-2.0 gdk-2.0
- (FIXED) bbb.default.Benchmark.netpipe - unpack error - missing netpipe-makefile.patch
- (TRANSIENT x4) bbb.default.Benchmark.netpipe - error write or reading synchronization string: Connection reset by peer
- (FIXED) bbb.default.Benchmark.STREAM - processing error - in parser.py in valid literal for float()
- bbb.default.Benchmark.x11perf - build error - configure error - missing package xmuu
- bbb.default.Functional.aiostress - runtime error - missing libaio.o on target
- bbb.default.Functional.ft2demos - build error - missing libXau and libXmcdp
- bbb.default.Functional.glib - build error - processor does not support Thumb mode
- bbb.default.Functional.kernel_build - build error - for scripts/mod/empty - unrecognized command line options to gcc
- bbb.default.Functional.LTP - aborted due to timeout of 40 minutes
- (FIXED) bbb.default.Functional.rmaptest - pre_test error - LINK issue
- ren1.default.Benchmark.blobksallad - build error - missing SDL.h
- ren1.default.Benchmark.Dhrystone - run error - measured time too small
- (FIXED) ren1.default.Benchmark.ffsb - build error - configure aarch64-poky not recognized
- ren1.default.Benchmark.GLMark - build error - missing SDL.h
- (FIXED) ren1.default.Benchmark.gtkperf - build error - configure aarch64-poky not recognized
- ren1.default.Benchmark.gtkperf - run error - gdk-pixbuf-query-loaders: command not found
- (FIXED) ren1.default.Benchmark.iperf - build error - configure machine aarch64-poky not recognized
- (FIXED) ren1.default.Benchmark.netperf - build error - configure machine aarch64-poky not recognized
- (FIXED) ren1.default.Benchmark.netpipe - unpack error
- (FIXED) ren1.default.Benchmark.Stream - processing error - parser.py invalid literal for float()
- ren1.default.Benchmark.tiobench - build error - crc32 compilation problem
- ren1.default.Benchmark.x11perf - build error - configure error - missing package xmuu
- ren1.default.Functional.aiostress - build error - missing libaio.h
- ren1.default.Functional.crashme - runtime error - only got 507 but was expecting 3000
- (FIXED) ren1.default.Functional.expat - build error - configure machine aarch64-poky not recognized
- (FIXED) ren1.default.Functional.expat - build error - cast loses permission in expat.h:674
- (FIXED) ren1.default.Functional.ft2demos - build error - configure machine aarch64-poky not recognized
- ren1.default.Functional.ft2demos - build error - cannot find library =/usr/lib/libxcb.la
- (FIXED) ren1.default.Functional.glib - build error - configure machine aarch64-poky not recognized
- ren1.default.Functional.glib - run error - exits with 143
- ren1.default.Functional.kernel_build - build error - unrecognized command line options to gcc
- (LAB ERROR) ren1.default.Functional.LTP - didn't build - target was down
- ren1.default.Functional.LTP - 'Rebuild' flag is true after job creation
- (FIXED) ren1.default.Functional.netperf - build error - configure machine aarch64-poky not recognized
- ren1.default.Functional.OpenSSL - build error - invalid option to make
- ren1.default.Functional.scrashme - build error - NR_SYSCALLS undeclared
- min1.default.Benchmark.blobsallad - build error - missing SDL.h
- min1.default.Benchmark.GLMark - build error - missing SDL.h
- min1.default.Benchmark.gtkperf - build error - configure dies checking for gtk+
- (FIXED) min1.default.Benchmark.lmbench2 - run error - command not found
- (FIXED) min1.default.Benchmark.lmbench2 - 'Rebuild' flag is true after job creation
- (FIXED) min1.default.Benchmark.netpipe - unpack error
- (TRANSIENT x2) min1.default.Benchmark.netpipe - error write or reading synchronization string: Connection reset by peer
- min1.default.Benchmark.OpenSSL - run error - core dump doing 512-bit private rsa
- min1.default.Benchmark.reboot - run error - can't cd to /tmp test dir.
- (FIXED) min1.default.Benchmark.Stream - processing - parser.py fault
- (FIXED) min1.default.Benchmark.x11perf - build error - missing package xmuu
- min1.default.Benchmark.x11perf - run error - unable to open display ':0'
- min1.default.Functional.aiostress - runtime error - missing libaio.so.1
- (FIXED) min1.default.Functional.expat - build error - cast loses precision in expat.h:674
- min1.default.Functional.glib - run error - program returns 143
- min1.default.Functional.LTP - 'Rebuild' flag is true after job creation
- min1.default.Functional.LTP - timeout - had 40m timeout
- min1.default.Functional.OpenSSL - timeout - had 6m timeout
- docker.bc-by2.Functional.bc - run error - FAIL
- docker.default.Benchmark.blobsallas - pretest - XRANDR not defined (ok for 1.2)
- docker.default.Benchmark.cyclictest - run error - needs to run as root!
- docker.default.Benchmark.dbench - run error - problem in hd_test_mount_prepare (/a)
- docker.default.Benchmark.Dhrystone - run error - measured time too small for meaningful results
- docker.default.Benchmark.lmbench2 - processing - parser.py parsing Local_Communcation_latences.AF_UNIX
- docker.default.Benchmark.OpenSSL - run error - Segfault! - maybe real error?
- (FIXED) docker.default.Benchmark.signaltest - run error - needs to run as root!
- docker.default.Functional.glib - processing - returns exit code 143
- docker.default.Functional.LTP - processing - results did not pass threshold
- docker.docker.Functional.LTP - load - creat05, fork09 took very long (I manually killed them)
- docker.default.Functional.OpenSSL - run error - tiemout (only had 6m)
- docker.default.Functional.tiff - build error - missing libtiff (caught by dependency check)
- (FIXED) min1.default.Benchmark.netpipe - jenkins view - there are no charts!
- there are no measures defined for this test
- this test had no parser.py
- 'ftc gen-report --where test=Benchmark.Interbench' is empty
- ftc gen-report --where test=Interbench' works
- top level result from testlog, when test fails, is not put into run.json
- there should be status=ERROR for test, with abort message for test when the run phase doesn't finish cleanly, or parser.py dies during processing
table of current test status [edit section]
If a test is not listed here, it passes on all 4 platforms (bbb, ren1, min1, docker)
| test ^ | bbb ^ | ren1 ^ | min1 ^ | docker ^ | notes ^ | disposition ^ |
| Functional.LTP | . | criteria failure | . | creat05, fork09 problems (high-CPU zombie, even after manual kill) | (with lengthened timeout, spec=default) | won't fix for 1.2 |
| Benchmark.ffsb | ok | fixed | ok | ok | . | ok for 1.2 |
| Functional.glib | processor does not support Thumb mode | program returns 143 exit code | program returns 143 exit code | . | . | won't fix for 1.2 (high priority) |
| Benchmark.x11perf | configure error - xmuu not found | configure error - xmuu not found | can't open display ':0' | . | . | won't fix for 1.2 |
| Functional.aiostress | missing libaio.o on target | missing libaio.h | missing libaio.so.1 on target | . | . | won't fix for 1.2 |
| Benchmark.Stream | fixed | fixed | fixed | . | . | ok for 1.2 |
| Benchmark.tiobench | ok | crc32 build error | ok | . | . | won't fix for 1.2 |
| Benchmark.reboot | ok | . | run error can't cd to /tmp test dir | not applicable | . | won't fix for 1.2 |
| Benchmark.netperf | ok | fixed | ok | . | . | ok for 1.2 |
| Benchmark.netpipe | fixed | fixed | fixed | . | . | ok for 1.2 |
| Functional.rmaptest | ok | ok | ok | . | . | ok for 1.2 |
| Functional.crashme | ok | expected 3000, got 507 | ok | . | . | won't fix for 1.2 |
| Functional.expat | ok | fixed | fixed | ok | . | ok for 1.2 |
| Functional.netperf | ok | fixed | ok | . | . | ok for 1.2 |
| Functional.kernel_build | for scripts/mod/empty - unrecognized command line options to gcc | unrecognized command line options to gcc | ok | . | . | needs review |
| Functional.ft2demos | missing libXau and libXmcdp | missing libscb.la | ok | . | . | won't fix for 1.2 |
| Benchmark.blobsallad | missing SDL.h | missing SDL.h | can't open display :0 | missing xrandr | . | document build problems |
| Benchmark.Dhrystone | ok | measured time too small (spec=default) | ok | measure time too small (spec=default) | . | ok for 1.2 |
| Benchmark.dbench | ok | ok | ok | problem with hd_test_mount_prepare (spec=default), with spec=testdir, it's OK | . | ok for 1.2 |
| Functional.scrashme | ok | NR_SYSCALLS undeclared | ok | . | . | won't fix for 1.2 |
| Benchmark.lmbench2 | ok | ok | fixed | parser.py error | . | ok for 1.2 |
| Benchmark.iperf | ok | fixed | ok | . | . | ok for 1.2 |
| Benchmark.gtkperf | configure error checking gtk+ | gdk-pixbuf-query-loaders: command not found | configure dies checking gtk+ | . | . | won't fix for 1.2 |
| Benchmark.GLmark | missing SDL.h | missing SDL.h | struct member is private in vector.h | . | Should provide instructions for adding libsdl to yocto build, or build it ourselves?? - actually, maybe should switch to newer glmark2, which tests OpenGL performance | won't fix for 1.2 |
| Functional.LTP | fixed | fixed | fixed | . | default timeout is 40m (too short for some boards, when build is included) | ok for 1.2 |
| Functional.OpenSSL | ok | invalid option to make | timeout | . | default timeout is 6m (too short) | won't fix for 1.2 |
| Benchmark.Interbench | test hangs on target | ok | ok | . | demonstrates a real bug with bbb | ok for 1.2 |
| Benchmark.signaltest | ok | ok | ok | fixed | docker needs root permissions | ok for 1.2 |
original problems by test and board [edit section]
| test ^ | bbb ^ | ren1 ^ | min1 ^ | docker ^ | notes ^ |
| Benchmark.netperf | ok | configure: machine aarch64-poky not recognized | ok | ||
| Functional.LTP | . | criteria failure | . | . | (with lengthened timeout, spec=default) |
| Benchmark.GLmark | missing SDL.h | missing SDL.h | missing SDL.h | . | . |
| Functional.kernel_build | for scripts/mod/empty - unrecognized command line options to gcc | unrecognized command line options to gcc | ok | . | . |
| Functional.aiostress | missing libaio.o on target | missing libaio.h | missing libaio.so.1 on target | . | . |
| Functional.crashme | ok | expected 3000, got 507 | ok | . | . |
| Benchmark.tiobench | ok | crc32 build error | ok | . | . |
| Benchmark.x11perf | configure error - xmuu not found | configure error - xmuu not found | missing xmuu | . | . |
| Benchmark.Stream | parser.py exception | parser.py exception | parser.py exception | . | . |
| Benchmark.netpipe | unpack error, missing plot | unpack error, missing plot | unpack error, missing plot | . | . |
| Benchmark.reboot | ok | . | run error can't cd to /tmp test dir | . | . |
| Functional.scrashme | ok | NR_SYSCALLS undeclared | ok | . | . |
| Functional.expat | ok | configure: machine aarch64-poky not recognized, cast loses precision in expat.h:674 | cast loses precision in expat.h:674 | . | . |
| Functional.ft2demos | missing libXau and libXmcdp | configure: machine aarch64-poky not recognized | ok | . | . |
| Functional.netperf | ok | configure: machine aarch64-poky not recognized | ok | . | . |
| Benchmark.gtkperf | configure error checking gtk+ | configure: machine aarch64-poky not recognized | configure dies checking gtk+ | . | . |
| Benchmark.iperf | ok | configure: machine aarch64-poky not recognized | ok | . | . |
| Functional.glib | processor does not support Thumb mode | configure: machine aarch64-poky not recognized | program returns 143 exit code | . | . |
| Benchmark.blobsallad | missing SDL.h | missing SDL.h | missing SDL.h | . | . |
| Functional.rmaptest | ok | ok | ok | . | . |
| Benchmark.ffsb | ok | configure: machine aarch64-poky not recognized | ok | . | . |
| Benchmark.Dhrystone | ok | measured time too small (spec=default) | ok | . | . |
| Benchmark.lmbench2 | ok | ok | run error - command not found | . | . |
| Benchmark.Interbench | test hangs on target | ok | ok | . | . |
| Functional.LTP | timeout | (transient failure) | timeout | creat05, fork09 problems (high-CPU zombie, even after manual kill) | default timeout is 40m (too short for some boards, when build is included) |
| Functional.OpenSSL | ok | invalid option to make | timeout | . | default timeout is 6m (too short) |
| Benchmark.signaltest | ok | ok | ok | root permissions checked in run phase | docker needs root permissions |