release-1.2-test-2017-09-27_Notes 

Fuego wiki

Login or create account

release-1.2-test-2017-09-27 Notes

Here are some notes on the 1.2 release:

flat problem list [edit section]

Resolution key:
  • FIXED - problem fixed with change in Fuego
  • TRANSIENT - problem went away with subsequent test and no change in Fuego
  • LAB ERROR - problem was with setup, config, or board status, issue can be ignored

Bugs seen:

  • bbb.default.Benchmark.Interbench - aborted due to timeout of 40 minutes
    • hangs in simulated CPU audio test - calloc cannot allocate memory
    • leaves 10 zombies on the target
  • bbb.default.Benchmark.Interbench - had Rebuild set to 'true' after job creation
  • bbb - lots of tests failed because bbb wedged after a test
    • (should look at history and see which test ran last on 9/27)
  • bbb.default.Benchmark.blobsallad - build error - missing SDL.h
  • bbb.default.Benchmark.GLMark - build error - missing SDL.h
  • bbb.default.Benchmark.gtkperf - build error - configure error checking for gtk+-2.0 gdk-2.0
  • (FIXED) bbb.default.Benchmark.netpipe - unpack error - missing netpipe-makefile.patch
  • (TRANSIENT x4) bbb.default.Benchmark.netpipe - error write or reading synchronization string: Connection reset by peer
  • (FIXED) bbb.default.Benchmark.STREAM - processing error - in parser.py in valid literal for float()
  • bbb.default.Benchmark.x11perf - build error - configure error - missing package xmuu
  • bbb.default.Functional.aiostress - runtime error - missing libaio.o on target
  • bbb.default.Functional.ft2demos - build error - missing libXau and libXmcdp
  • bbb.default.Functional.glib - build error - processor does not support Thumb mode
  • bbb.default.Functional.kernel_build - build error - for scripts/mod/empty - unrecognized command line options to gcc
  • bbb.default.Functional.LTP - aborted due to timeout of 40 minutes
  • (FIXED) bbb.default.Functional.rmaptest - pre_test error - LINK issue
  • ren1.default.Benchmark.blobksallad - build error - missing SDL.h
  • ren1.default.Benchmark.Dhrystone - run error - measured time too small
  • (FIXED) ren1.default.Benchmark.ffsb - build error - configure aarch64-poky not recognized
  • ren1.default.Benchmark.GLMark - build error - missing SDL.h
  • (FIXED) ren1.default.Benchmark.gtkperf - build error - configure aarch64-poky not recognized
  • ren1.default.Benchmark.gtkperf - run error - gdk-pixbuf-query-loaders: command not found
  • (FIXED) ren1.default.Benchmark.iperf - build error - configure machine aarch64-poky not recognized
  • (FIXED) ren1.default.Benchmark.netperf - build error - configure machine aarch64-poky not recognized
  • (FIXED) ren1.default.Benchmark.netpipe - unpack error
  • (FIXED) ren1.default.Benchmark.Stream - processing error - parser.py invalid literal for float()
  • ren1.default.Benchmark.tiobench - build error - crc32 compilation problem
  • ren1.default.Benchmark.x11perf - build error - configure error - missing package xmuu
  • ren1.default.Functional.aiostress - build error - missing libaio.h
  • ren1.default.Functional.crashme - runtime error - only got 507 but was expecting 3000
  • (FIXED) ren1.default.Functional.expat - build error - configure machine aarch64-poky not recognized
  • (FIXED) ren1.default.Functional.expat - build error - cast loses permission in expat.h:674
  • (FIXED) ren1.default.Functional.ft2demos - build error - configure machine aarch64-poky not recognized
  • ren1.default.Functional.ft2demos - build error - cannot find library =/usr/lib/libxcb.la
  • (FIXED) ren1.default.Functional.glib - build error - configure machine aarch64-poky not recognized
  • ren1.default.Functional.glib - run error - exits with 143
  • ren1.default.Functional.kernel_build - build error - unrecognized command line options to gcc
  • (LAB ERROR) ren1.default.Functional.LTP - didn't build - target was down
  • ren1.default.Functional.LTP - 'Rebuild' flag is true after job creation
  • (FIXED) ren1.default.Functional.netperf - build error - configure machine aarch64-poky not recognized
  • ren1.default.Functional.OpenSSL - build error - invalid option to make
  • ren1.default.Functional.scrashme - build error - NR_SYSCALLS undeclared
  • min1.default.Benchmark.blobsallad - build error - missing SDL.h
  • min1.default.Benchmark.GLMark - build error - missing SDL.h
  • min1.default.Benchmark.gtkperf - build error - configure dies checking for gtk+
  • (FIXED) min1.default.Benchmark.lmbench2 - run error - command not found
  • (FIXED) min1.default.Benchmark.lmbench2 - 'Rebuild' flag is true after job creation
  • (FIXED) min1.default.Benchmark.netpipe - unpack error
  • (TRANSIENT x2) min1.default.Benchmark.netpipe - error write or reading synchronization string: Connection reset by peer
  • min1.default.Benchmark.OpenSSL - run error - core dump doing 512-bit private rsa
  • min1.default.Benchmark.reboot - run error - can't cd to /tmp test dir.
  • (FIXED) min1.default.Benchmark.Stream - processing - parser.py fault
  • (FIXED) min1.default.Benchmark.x11perf - build error - missing package xmuu
  • min1.default.Benchmark.x11perf - run error - unable to open display ':0'
  • min1.default.Functional.aiostress - runtime error - missing libaio.so.1
  • (FIXED) min1.default.Functional.expat - build error - cast loses precision in expat.h:674
  • min1.default.Functional.glib - run error - program returns 143
  • min1.default.Functional.LTP - 'Rebuild' flag is true after job creation
  • min1.default.Functional.LTP - timeout - had 40m timeout
  • min1.default.Functional.OpenSSL - timeout - had 6m timeout
  • docker.bc-by2.Functional.bc - run error - FAIL
  • docker.default.Benchmark.blobsallas - pretest - XRANDR not defined (ok for 1.2)
  • docker.default.Benchmark.cyclictest - run error - needs to run as root!
  • docker.default.Benchmark.dbench - run error - problem in hd_test_mount_prepare (/a)
  • docker.default.Benchmark.Dhrystone - run error - measured time too small for meaningful results
  • docker.default.Benchmark.lmbench2 - processing - parser.py parsing Local_Communcation_latences.AF_UNIX
  • docker.default.Benchmark.OpenSSL - run error - Segfault! - maybe real error?
  • (FIXED) docker.default.Benchmark.signaltest - run error - needs to run as root!
  • docker.default.Functional.glib - processing - returns exit code 143
  • docker.default.Functional.LTP - processing - results did not pass threshold
  • docker.docker.Functional.LTP - load - creat05, fork09 took very long (I manually killed them)
  • docker.default.Functional.OpenSSL - run error - tiemout (only had 6m)
  • docker.default.Functional.tiff - build error - missing libtiff (caught by dependency check)

  • (FIXED) min1.default.Benchmark.netpipe - jenkins view - there are no charts!
    • there are no measures defined for this test
    • this test had no parser.py

  • 'ftc gen-report --where test=Benchmark.Interbench' is empty
    • ftc gen-report --where test=Interbench' works
  • top level result from testlog, when test fails, is not put into run.json
    • there should be status=ERROR for test, with abort message for test when the run phase doesn't finish cleanly, or parser.py dies during processing

table of current test status [edit section]

If a test is not listed here, it passes on all 4 platforms (bbb, ren1, min1, docker)

test  ^ bbb  ^ ren1  ^ min1  ^ docker  ^ notes  ^ disposition  ^
Functional.LTP . criteria failure . creat05, fork09 problems (high-CPU zombie, even after manual kill) (with lengthened timeout, spec=default) won't fix for 1.2
Benchmark.ffsb ok fixed ok ok . ok for 1.2
Functional.glib processor does not support Thumb mode program returns 143 exit code program returns 143 exit code . . won't fix for 1.2 (high priority)
Benchmark.x11perf configure error - xmuu not found configure error - xmuu not found can't open display ':0' . . won't fix for 1.2
Functional.aiostress missing libaio.o on target missing libaio.h missing libaio.so.1 on target . . won't fix for 1.2
Benchmark.Stream fixed fixed fixed . . ok for 1.2
Benchmark.tiobench ok crc32 build error ok . . won't fix for 1.2
Benchmark.reboot ok . run error can't cd to /tmp test dir not applicable . won't fix for 1.2
Benchmark.netperf ok fixed ok . . ok for 1.2
Benchmark.netpipe fixed fixed fixed . . ok for 1.2
Functional.rmaptest ok ok ok . . ok for 1.2
Functional.crashme ok expected 3000, got 507 ok . . won't fix for 1.2
Functional.expat ok fixed fixed ok . ok for 1.2
Functional.netperf ok fixed ok . . ok for 1.2
Functional.kernel_build for scripts/mod/empty - unrecognized command line options to gcc unrecognized command line options to gcc ok . . needs review
Functional.ft2demos missing libXau and libXmcdp missing libscb.la ok . . won't fix for 1.2
Benchmark.blobsallad missing SDL.h missing SDL.h can't open display :0 missing xrandr . document build problems
Benchmark.Dhrystone ok measured time too small (spec=default) ok measure time too small (spec=default) . ok for 1.2
Benchmark.dbench ok ok ok problem with hd_test_mount_prepare (spec=default), with spec=testdir, it's OK . ok for 1.2
Functional.scrashme ok NR_SYSCALLS undeclared ok . . won't fix for 1.2
Benchmark.lmbench2 ok ok fixed parser.py error . ok for 1.2
Benchmark.iperf ok fixed ok . . ok for 1.2
Benchmark.gtkperf configure error checking gtk+ gdk-pixbuf-query-loaders: command not found configure dies checking gtk+ . . won't fix for 1.2
Benchmark.GLmark missing SDL.h missing SDL.h struct member is private in vector.h . Should provide instructions for adding libsdl to yocto build, or build it ourselves?? - actually, maybe should switch to newer glmark2, which tests OpenGL performance won't fix for 1.2
Functional.LTP fixed fixed fixed . default timeout is 40m (too short for some boards, when build is included) ok for 1.2
Functional.OpenSSL ok invalid option to make timeout . default timeout is 6m (too short) won't fix for 1.2
Benchmark.Interbench test hangs on target ok ok . demonstrates a real bug with bbb ok for 1.2
Benchmark.signaltest ok ok ok fixed docker needs root permissions ok for 1.2
Query

original problems by test and board [edit section]

test  ^ bbb  ^ ren1  ^ min1  ^ docker  ^ notes  ^
Benchmark.netperf ok configure: machine aarch64-poky not recognized ok
Functional.LTP . criteria failure . . (with lengthened timeout, spec=default)
Benchmark.GLmark missing SDL.h missing SDL.h missing SDL.h . .
Functional.kernel_build for scripts/mod/empty - unrecognized command line options to gcc unrecognized command line options to gcc ok . .
Functional.aiostress missing libaio.o on target missing libaio.h missing libaio.so.1 on target . .
Functional.crashme ok expected 3000, got 507 ok . .
Benchmark.tiobench ok crc32 build error ok . .
Benchmark.x11perf configure error - xmuu not found configure error - xmuu not found missing xmuu . .
Benchmark.Stream parser.py exception parser.py exception parser.py exception . .
Benchmark.netpipe unpack error, missing plot unpack error, missing plot unpack error, missing plot . .
Benchmark.reboot ok . run error can't cd to /tmp test dir . .
Functional.scrashme ok NR_SYSCALLS undeclared ok . .
Functional.expat ok configure: machine aarch64-poky not recognized, cast loses precision in expat.h:674 cast loses precision in expat.h:674 . .
Functional.ft2demos missing libXau and libXmcdp configure: machine aarch64-poky not recognized ok . .
Functional.netperf ok configure: machine aarch64-poky not recognized ok . .
Benchmark.gtkperf configure error checking gtk+ configure: machine aarch64-poky not recognized configure dies checking gtk+ . .
Benchmark.iperf ok configure: machine aarch64-poky not recognized ok . .
Functional.glib processor does not support Thumb mode configure: machine aarch64-poky not recognized program returns 143 exit code . .
Benchmark.blobsallad missing SDL.h missing SDL.h missing SDL.h . .
Functional.rmaptest ok ok ok . .
Benchmark.ffsb ok configure: machine aarch64-poky not recognized ok . .
Benchmark.Dhrystone ok measured time too small (spec=default) ok . .
Benchmark.lmbench2 ok ok run error - command not found . .
Benchmark.Interbench test hangs on target ok ok . .
Functional.LTP timeout (transient failure) timeout creat05, fork09 problems (high-CPU zombie, even after manual kill) default timeout is 40m (too short for some boards, when build is included)
Functional.OpenSSL ok invalid option to make timeout . default timeout is 6m (too short)
Benchmark.signaltest ok ok ok root permissions checked in run phase docker needs root permissions
Query

TBWiki engine 1.8.3 by Tim Bird