release-1.2-test-2017-09-27_Notes 

Fuego wiki

Login or create account

release-1.2-test-2017-09-27 Notes

Here are some notes on the 1.2 release:

flat problem list [edit section]

Resolution key:
  • FIXED - problem fixed with change in Fuego
  • TRANSIENT - problem went away with subsequent test and no change in Fuego
  • LAB ERROR - problem was with setup, config, or board status, issue can be ignored

Bugs seen:

  • bbb.default.Benchmark.Interbench - aborted due to timeout of 40 minutes
    • hangs in simulated CPU audio test - calloc cannot allocate memory
    • leaves 10 zombies on the target
  • bbb.default.Benchmark.Interbench - had Rebuild set to 'true' after job creation
  • bbb - lots of tests failed because bbb wedged after a test
    • (should look at history and see which test ran last on 9/27)
  • bbb.default.Benchmark.blobsallad - build error - missing SDL.h
  • bbb.default.Benchmark.GLMark - build error - missing SDL.h
  • bbb.default.Benchmark.gtkperf - build error - configure error checking for gtk+-2.0 gdk-2.0
  • (FIXED) bbb.default.Benchmark.netpipe - unpack error - missing netpipe-makefile.patch
  • (TRANSIENT x4) bbb.default.Benchmark.netpipe - error write or reading synchronization string: Connection reset by peer
  • (FIXED) bbb.default.Benchmark.STREAM - processing error - in parser.py in valid literal for float()
  • bbb.default.Benchmark.x11perf - build error - configure error - missing package xmuu
  • bbb.default.Functional.aiostress - runtime error - missing libaio.o on target
  • bbb.default.Functional.ft2demos - build error - missing libXau and libXmcdp
  • bbb.default.Functional.glib - build error - processor does not support Thumb mode
  • bbb.default.Functional.kernel_build - build error - for scripts/mod/empty - unrecognized command line options to gcc
  • bbb.default.Functional.LTP - aborted due to timeout of 40 minutes
  • (FIXED) bbb.default.Functional.rmaptest - pre_test error - LINK issue
  • ren1.default.Benchmark.blobksallad - build error - missing SDL.h
  • ren1.default.Benchmark.Dhrystone - run error - measured time too small
  • (FIXED) ren1.default.Benchmark.ffsb - build error - configure aarch64-poky not recognized
  • ren1.default.Benchmark.GLMark - build error - missing SDL.h
  • (FIXED) ren1.default.Benchmark.gtkperf - build error - configure aarch64-poky not recognized
  • ren1.default.Benchmark.gtkperf - run error - gdk-pixbuf-query-loaders: command not found
  • (FIXED) ren1.default.Benchmark.iperf - build error - configure machine aarch64-poky not recognized
  • (FIXED) ren1.default.Benchmark.netperf - build error - configure machine aarch64-poky not recognized
  • (FIXED) ren1.default.Benchmark.netpipe - unpack error
  • (FIXED) ren1.default.Benchmark.Stream - processing error - parser.py invalid literal for float()
  • ren1.default.Benchmark.tiobench - build error - crc32 compilation problem
  • ren1.default.Benchmark.x11perf - build error - configure error - missing package xmuu
  • ren1.default.Functional.aiostress - build error - missing libaio.h
  • ren1.default.Functional.crashme - runtime error - only got 507 but was expecting 3000
  • (FIXED) ren1.default.Functional.expat - build error - configure machine aarch64-poky not recognized
  • (FIXED) ren1.default.Functional.expat - build error - cast loses permission in expat.h:674
  • (FIXED) ren1.default.Functional.ft2demos - build error - configure machine aarch64-poky not recognized
  • ren1.default.Functional.ft2demos - build error - cannot find library =/usr/lib/libxcb.la
  • (FIXED) ren1.default.Functional.glib - build error - configure machine aarch64-poky not recognized
  • ren1.default.Functional.glib - run error - exits with 143
  • ren1.default.Functional.kernel_build - build error - unrecognized command line options to gcc
  • (LAB ERROR) ren1.default.Functional.LTP - didn't build - target was down
  • ren1.default.Functional.LTP - 'Rebuild' flag is true after job creation
  • (FIXED) ren1.default.Functional.netperf - build error - configure machine aarch64-poky not recognized
  • ren1.default.Functional.OpenSSL - build error - invalid option to make
  • ren1.default.Functional.scrashme - build error - NR_SYSCALLS undeclared
  • min1.default.Benchmark.blobsallad - build error - missing SDL.h
  • min1.default.Benchmark.GLMark - build error - missing SDL.h
  • min1.default.Benchmark.gtkperf - build error - configure dies checking for gtk+
  • (FIXED) min1.default.Benchmark.lmbench2 - run error - command not found
  • (FIXED) min1.default.Benchmark.lmbench2 - 'Rebuild' flag is true after job creation
  • (FIXED) min1.default.Benchmark.netpipe - unpack error
  • (TRANSIENT x2) min1.default.Benchmark.netpipe - error write or reading synchronization string: Connection reset by peer
  • min1.default.Benchmark.OpenSSL - run error - core dump doing 512-bit private rsa
  • min1.default.Benchmark.reboot - run error - can't cd to /tmp test dir.
  • (FIXED) min1.default.Benchmark.Stream - processing - parser.py fault
  • (FIXED) min1.default.Benchmark.x11perf - build error - missing package xmuu
  • min1.default.Benchmark.x11perf - run error - unable to open display ':0'
  • min1.default.Functional.aiostress - runtime error - missing libaio.so.1
  • (FIXED) min1.default.Functional.expat - build error - cast loses precision in expat.h:674
  • min1.default.Functional.glib - run error - program returns 143
  • min1.default.Functional.LTP - 'Rebuild' flag is true after job creation
  • min1.default.Functional.LTP - timeout - had 40m timeout
  • min1.default.Functional.OpenSSL - timeout - had 6m timeout
  • docker.bc-by2.Functional.bc - run error - FAIL
  • docker.default.Benchmark.blobsallas - pretest - XRANDR not defined (ok for 1.2)
  • docker.default.Benchmark.cyclictest - run error - needs to run as root!
  • docker.default.Benchmark.dbench - run error - problem in hd_test_mount_prepare (/a)
  • docker.default.Benchmark.Dhrystone - run error - measured time too small for meaningful results
  • docker.default.Benchmark.lmbench2 - processing - parser.py parsing Local_Communcation_latences.AF_UNIX
  • docker.default.Benchmark.OpenSSL - run error - Segfault! - maybe real error?
  • (FIXED) docker.default.Benchmark.signaltest - run error - needs to run as root!
  • docker.default.Functional.glib - processing - returns exit code 143
  • docker.default.Functional.LTP - processing - results did not pass threshold
  • docker.docker.Functional.LTP - load - creat05, fork09 took very long (I manually killed them)
  • docker.default.Functional.OpenSSL - run error - tiemout (only had 6m)
  • docker.default.Functional.tiff - build error - missing libtiff (caught by dependency check)

  • (FIXED) min1.default.Benchmark.netpipe - jenkins view - there are no charts!
    • there are no measures defined for this test
    • this test had no parser.py

  • 'ftc gen-report --where test=Benchmark.Interbench' is empty
    • ftc gen-report --where test=Interbench' works
  • top level result from testlog, when test fails, is not put into run.json
    • there should be status=ERROR for test, with abort message for test when the run phase doesn't finish cleanly, or parser.py dies during processing

table of current test status [edit section]

If a test is not listed here, it passes on all 4 platforms (bbb, ren1, min1, docker)

test  ^ bbb  ^ ren1  ^ min1  ^ docker  ^ notes  ^ disposition  ^
Functional.LTP . criteria failure . creat05, fork09 problems (high-CPU zombie, even after manual kill) (with lengthened timeout, spec=default) won't fix for 1.2
Benchmark.x11perf configure error - xmuu not found configure error - xmuu not found can't open display ':0' . . won't fix for 1.2
Benchmark.gtkperf configure error checking gtk+ gdk-pixbuf-query-loaders: command not found configure dies checking gtk+ . . won't fix for 1.2
Functional.LTP fixed fixed fixed . default timeout is 40m (too short for some boards, when build is included) ok for 1.2
Benchmark.Stream fixed fixed fixed . . ok for 1.2
Benchmark.netpipe fixed fixed fixed . . ok for 1.2
Functional.kernel_build for scripts/mod/empty - unrecognized command line options to gcc unrecognized command line options to gcc ok . . needs review
Benchmark.GLmark missing SDL.h missing SDL.h struct member is private in vector.h . Should provide instructions for adding libsdl to yocto build, or build it ourselves?? - actually, maybe should switch to newer glmark2, which tests OpenGL performance won't fix for 1.2
Benchmark.blobsallad missing SDL.h missing SDL.h can't open display :0 missing xrandr . document build problems
Functional.ft2demos missing libXau and libXmcdp missing libscb.la ok . . won't fix for 1.2
Functional.aiostress missing libaio.o on target missing libaio.h missing libaio.so.1 on target . . won't fix for 1.2
Benchmark.ffsb ok fixed ok ok . ok for 1.2
Benchmark.tiobench ok crc32 build error ok . . won't fix for 1.2
Benchmark.reboot ok . run error can't cd to /tmp test dir not applicable . won't fix for 1.2
Benchmark.signaltest ok ok ok fixed docker needs root permissions ok for 1.2
Benchmark.netperf ok fixed ok . . ok for 1.2
Functional.rmaptest ok ok ok . . ok for 1.2
Functional.crashme ok expected 3000, got 507 ok . . won't fix for 1.2
Functional.expat ok fixed fixed ok . ok for 1.2
Functional.netperf ok fixed ok . . ok for 1.2
Functional.OpenSSL ok invalid option to make timeout . default timeout is 6m (too short) won't fix for 1.2
Benchmark.Dhrystone ok measured time too small (spec=default) ok measure time too small (spec=default) . ok for 1.2
Benchmark.dbench ok ok ok problem with hd_test_mount_prepare (spec=default), with spec=testdir, it's OK . ok for 1.2
Functional.scrashme ok NR_SYSCALLS undeclared ok . . won't fix for 1.2
Benchmark.lmbench2 ok ok fixed parser.py error . ok for 1.2
Benchmark.iperf ok fixed ok . . ok for 1.2
Functional.glib processor does not support Thumb mode program returns 143 exit code program returns 143 exit code . . won't fix for 1.2 (high priority)
Benchmark.Interbench test hangs on target ok ok . demonstrates a real bug with bbb ok for 1.2
Query

original problems by test and board [edit section]

test  ^ bbb  ^ ren1  ^ min1  ^ docker  ^ notes  ^
Functional.LTP . criteria failure . . (with lengthened timeout, spec=default)
Benchmark.x11perf configure error - xmuu not found configure error - xmuu not found missing xmuu . .
Benchmark.gtkperf configure error checking gtk+ configure: machine aarch64-poky not recognized configure dies checking gtk+ . .
Functional.kernel_build for scripts/mod/empty - unrecognized command line options to gcc unrecognized command line options to gcc ok . .
Benchmark.GLmark missing SDL.h missing SDL.h missing SDL.h . .
Benchmark.blobsallad missing SDL.h missing SDL.h missing SDL.h . .
Functional.ft2demos missing libXau and libXmcdp configure: machine aarch64-poky not recognized ok . .
Functional.aiostress missing libaio.o on target missing libaio.h missing libaio.so.1 on target . .
Functional.crashme ok expected 3000, got 507 ok . .
Benchmark.tiobench ok crc32 build error ok . .
Benchmark.signaltest ok ok ok root permissions checked in run phase docker needs root permissions
Benchmark.reboot ok . run error can't cd to /tmp test dir . .
Functional.scrashme ok NR_SYSCALLS undeclared ok . .
Functional.expat ok configure: machine aarch64-poky not recognized, cast loses precision in expat.h:674 cast loses precision in expat.h:674 . .
Functional.OpenSSL ok invalid option to make timeout . default timeout is 6m (too short)
Functional.netperf ok configure: machine aarch64-poky not recognized ok . .
Benchmark.iperf ok configure: machine aarch64-poky not recognized ok . .
Functional.rmaptest ok ok ok . .
Benchmark.ffsb ok configure: machine aarch64-poky not recognized ok . .
Benchmark.Dhrystone ok measured time too small (spec=default) ok . .
Benchmark.netperf ok configure: machine aarch64-poky not recognized ok
Benchmark.lmbench2 ok ok run error - command not found . .
Benchmark.Stream parser.py exception parser.py exception parser.py exception . .
Functional.glib processor does not support Thumb mode configure: machine aarch64-poky not recognized program returns 143 exit code . .
Benchmark.Interbench test hangs on target ok ok . .
Functional.LTP timeout (transient failure) timeout creat05, fork09 problems (high-CPU zombie, even after manual kill) default timeout is 40m (too short for some boards, when build is included)
Benchmark.netpipe unpack error, missing plot unpack error, missing plot unpack error, missing plot . .
Query

TBWiki engine 1.8.2 by Tim Bird