Release 1.3 testing - 2018-05-01 in 'raw' format
{{TableOfContents}}
Here are some notes on testing the 1.3 release (using the 412-container):
= flat problem list =
Resolution key:
- FIXED - problem fixed with change in Fuego
- TRANSIENT - problem went away with subsequent test and no change in Fuego
- LAB ERROR - problem was with setup, config, or board status, issue can be ignored
Bugs seen:
Here are testplan_lts failures for 2018-05-01
== min1 ==
- min1.default.Functional.aiostress - FAIL - missing shared lib libaio.so.1 (not new)
- error while loading shared libraries: libaio.so.1: cannot open shared object file: No such file or directory
- min1.default.Functional.glib - FAIL - exits with error code 143 (not new, high priority)
- min1.default.Functional.LTP - FAIL - 9 testase failures (see details below)
- min1.default.Functional.kernel_build - ERROR (expected)
- can only run on docker
- min1.default.Benchmark.signaltest - FAILURE (expected - not RT kernel)
- results did not satisfy threshold
- min1.default.Benchmark.cyclictest - FAILURE (expected - not RT kernel)
- results did not satisfy threshold
== ren1 ==
- ren1.default.Functional.aiostress - ERROR missing libaio.h (not new)
- (build phase): aiostress.c:58:20: fatal error: libaio.h: No such file or directory
- ren1.default.Fucntional.crashme - FAIL (not new, should examine)
- results did not satisfy the threshold
- log_compare "0 ... 3000" (but test result was "0 ... 507")
- ren1.default.Functional.ft2demos - ERROR - missing lib, can't build ftview?
- libtool: link: cannot find the library `=/usr/lib/libxcb.la' or unhandled argument `=/usr/lib/libxcb.la'
Makefile:470: recipe for target '/fuego-rw/buildzone/ren1.default.Functional.ft2demos-poky-aarch64/ft2demos-2.3.6/bin/ftview' failed
- make: *** [/fuego-rw/buildzone/ren1.default.Functional.ft2demos-poky-aarch64/ft2demos-2.3.6/bin/ftview] Error 1
- (NOTE: Also had an error about missing program 'build_error', but that was fixed: /fuego-core/engine/tests/Functional.ft2demos/fuego_test.sh: line 16: build_error: command not found)
- ren1.default.Functional.glib - FAIL (not new)
- returned exit code 143
- ren1.default.Functional.LTP - FAIL - 35 FAILS or SKIPs
- see details below
- ren1.default.Functional.kernel_build - ERROR - ABORT
- can only run on docker
== bbb ==
- bbb.default.Funcational.aiostress - ERROR
- bbb.default.Functional.LTP - ERROR (critical)
- LTP test hangs the board!!
- testcase fanotify07 hangs the board
- why doesn't the test recover after the hang?
- ltp-pan should time out (it doesn't)
- fanotify07 threads should be killed (they aren't)
- Fuego should reboot the board on post-test (it doesn't)
- bbb.default.Functional.ft2demos - ERROR
- /usr/lib/gcc-cross/arm-linux-gnueabihf/6/../../../../arm-linux-gnueabihf/bin/ld: cannot find -lXau
- /usr/lib/gcc-cross/arm-linux-gnueabihf/6/../../../../arm-linux-gnueabihf/bin/ld: cannot find -lXdmcp
- collect2: error: ld returned 1 exit status
- Makefile:426: recipe for target '/fuego-rw/buildzone/bbb.default.Functional.ft2demos-debian-armhf/ft2demos-2.3.6/bin/ftlint' failed
- make: *** [/fuego-rw/buildzone/bbb.default.Functional.ft2demos-debian-armhf/ft2demos-2.3.6/bin/ftlint] Error 1
- bbb.default.Functional.glib - ERROR (instructions in Thumb mode)
- (build phase) /tmp/ccSPOtNr.s:65: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode
-
- bbb.default.Functional.kernel_build - ERROR (expected?)
- can only run in docker
- bbb.default.Benchmark.OpenSSL - ERROR (missing zlib zconf.h)
- (build phase) arm-linux-gnueabihf-gcc -I.. -I../.. -I../asn1 -I../evp -I../../include --sysroot=/ -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -c -o c_zlib.o c_zlib.c
- In file included from c_zlib.c:25:0:
- /usr/include/zlib.h:34:19: fatal error: zconf.h: No such file or directory
- #include "zconf.h"
== rpi3-1 ==
- rpi3-1.default.Functional.aiostress - ERROR
- rpi3-1.default.Benchmark.ffsb
- rpi3-1.default.Functional.glib - ERROR
- rpi3-1.default.Functional.f2demos
- rpi3-1.default.Functional.LTP
- rpi3-1.default.Functional.bc - ERROR - no BC program on board
= NOTES on test duration =
- bbb.default.Functional.linus_stress took 27 minutes on 2018-05-01
= Overview =
{{{
root@tlinux:/# ftc list-runs --where "start_time>yesterday,status!=PASS" | sort -k 3
Runs on this host:
where start_time value is '1.5250788e+12'
Test Started host:board Status
--------------------------- ---------------------- ------------------------------ ------
Functional.LTP 2018-05-01_01:18:20 fuegohost:bbb ERROR
Functional.aiostress 2018-05-01_01:55:25 fuegohost:bbb ERROR
Functional.crashme 2018-05-01_02:25:11 fuegohost:bbb ERROR
Functional.ft2demos 2018-05-01_02:43:04 fuegohost:bbb ERROR
Functional.glib 2018-05-01_02:54:57 fuegohost:bbb ERROR
Functional.kernel_build 2018-05-01_00:44:46 fuegohost:bbb ERROR
Functional.linus_stress 2018-05-01_03:24:50 fuegohost:bbb ERROR
Functional.netperf 2018-05-01_03:36:43 fuegohost:bbb ERROR
Functional.rmaptest 2018-05-01_03:48:36 fuegohost:bbb ERROR
Functional.synctest 2018-05-01_04:00:29 fuegohost:bbb ERROR
Functional.LTP 2018-05-01_00:43:58 fuegohost:min1 FAIL
Functional.aiostress 2018-05-01_01:34:38 fuegohost:min1 FAIL
Functional.glib 2018-05-01_01:37:38 fuegohost:min1 FAIL
Functional.kernel_build 2018-05-01_01:39:42 fuegohost:min1 ERROR
Functional.LTP 2018-05-01_01:01:23 fuegohost:ren1 FAIL
Functional.aiostress 2018-05-01_00:54:12 fuegohost:ren1 ERROR
Functional.crashme 2018-05-01_00:56:46 fuegohost:ren1 FAIL
Functional.ft2demos 2018-05-01_00:55:07 fuegohost:ren1 ERROR
Functional.glib 2018-05-01_00:49:26 fuegohost:ren1 FAIL
Functional.kernel_build 2018-05-01_00:47:49 fuegohost:ren1 ERROR
Benchmark.ffsb 2018-05-01_00:59:25 fuegohost:rpi3-1 SKIP
Functional.LTP 2018-05-01_01:02:42 fuegohost:rpi3-1 FAIL
Functional.aiostress 2018-05-01_00:44:39 fuegohost:rpi3-1 FAIL
Functional.bc 2018-05-01_02:35:33 fuegohost:rpi3-1 ERROR
Functional.ft2demos 2018-05-01_00:45:07 fuegohost:rpi3-1 ERROR
Functional.glib 2018-05-01_02:38:14 fuegohost:rpi3-1 ERROR
Functional.kernel_build 2018-05-01_01:01:38 fuegohost:rpi3-1 ERROR
--------------------------- ---------------------- ------------------------------ ------
}}}
= Details =
== min1 LTP testcase problems ==
{{{root@tlinux:/# ftc gen-report --where "board=min1,test=LTP,start_time>yesterday,tguid:result!=PASS"
where start_time value is '1.5250788e+12'
======================================================================
**** Fuego Test Report ****
test : LTP
board : min1
kernel : 4.13.0-39-generic
timestamp : 2018-05-01_00:43:58
report_date : 2018-05-01_17:55:27
======================================================================
--------------------------------------------------------------------------------------------------------------------------------
test_name spec board timestamp tguid tguid:result
--------------------------------------------------------------------------------------------------------------------------------
Functional.LTP default min1 2018-05-01_00:43:58 syscalls FAIL
Functional.LTP default min1 2018-05-01_00:43:58 syscalls.io_getevents01 FAIL
Functional.LTP default min1 2018-05-01_00:43:58 syscalls.fanotify06 FAIL
Functional.LTP default min1 2018-05-01_00:43:58 syscalls.io_submit01 FAIL
Functional.LTP default min1 2018-05-01_00:43:58 syscalls.io_cancel01 FAIL
Functional.LTP default min1 2018-05-01_00:43:58 syscalls.msgctl11 FAIL
Functional.LTP default min1 2018-05-01_00:43:58 syscalls.eventfd01 FAIL
Functional.LTP default min1 2018-05-01_00:43:58 syscalls.io_setup01 FAIL
Functional.LTP default min1 2018-05-01_00:43:58 syscalls.io_destroy01 FAIL
Functional.LTP default min1 2018-05-01_00:43:58 sem_init.sem_init_7-1 SKIP
--------------------------------------------------------------------------------------------------------------------------------
}}}
== ren1 LTP testcase problems ==
35 fails or skips
{{{
root@tlinux:/# ftc gen-report --where "board=ren1,test=LTP,start_time>yesterday,tguid:result!=PASS" --fields timestamp,build_number,kernel,tguid,tguid:result
where start_time value is '1.5250788e+12'
======================================================================
**** Fuego Test Report ****
test : LTP
board : ren1
kernel : 4.9.0-yocto-standard
timestamp : 2018-05-01_01:01:23
report_date : 2018-05-01_18:45:22
======================================================================
-----------------------------------------------------------------------------------------------------------
timestamp build_number kernel tguid tguid:result
-----------------------------------------------------------------------------------------------------------
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.cve-2017-6951 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.prot_hsymlinks FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.renameat202 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.add_key04 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.keyctl08 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.keyctl03 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.keyctl02 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.keyctl05 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.keyctl04 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.keyctl07 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.keyctl06 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.msgctl09 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.msgctl08 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.fanotify08 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.fanotify07 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.fanotify06 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.utime02 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.perf_event_open02 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.abort01 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.recvmsg01 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.kill11 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.msgctl10 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.msgctl11 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.request_key02 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.open10 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.renameat201 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.add_key03 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.setsockopt02 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.cve-2017-5669 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.sendmsg01 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.request_key01 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.request_key03 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.setpriority01 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard syscalls.utime01 FAIL
2018-05-01_01:01:23 3 4.9.0-yocto-standard sem_init.sem_init_7-1 SKIP
-----------------------------------------------------------------------------------------------------------
}}}
== bbb.default.Functional.glib build error details ==
{{{
libtool: compile: arm-linux-gnueabihf-gcc -DHAVE_CONFIG_H -I. -I.. -I.. \
-DG_LOG_DOMAIN=\"GLib\" -DG_DISABLE_CAST_CHECKS -DG_DISABLE_DEPRECATED \
-DGLIB_COMPILATION -DPCRE_STATIC -DG_DISABLE_SINGLE_INCLUDES -pthread \
-g -O2 -Wall -MT gatomic.lo -MD -MP -MF .deps/gatomic.Tpo -c gatomic.c \
-fPIC -DPIC -o .libs/gatomic.o
/tmp/ccSPOtNr.s: Assembler messages:
/tmp/ccSPOtNr.s:65: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode
/tmp/ccSPOtNr.s:85: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode
/tmp/ccSPOtNr.s:188: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode
/tmp/ccSPOtNr.s:208: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode
/tmp/ccSPOtNr.s:310: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode
/tmp/ccSPOtNr.s:330: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode
/tmp/ccSPOtNr.s:432: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode
/tmp/ccSPOtNr.s:452: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode
Makefile:994: recipe for target 'gatomic.lo' failed
}}}
== rpi3-11 LTP testcase problems ==
{{{
root@tlinux:/# ftc gen-report --where "board=rpi3-1,test=LTP,tguid:result!=PASS,start_time>2 days ago"
where start_time value is '1.525060103e+12'
======================================================================
**** Fuego Test Report ****
test : LTP
board : rpi3-1
kernel : 4.9.41-v7+
timestamp : 2018-05-01_01:02:42
report_date : 2018-05-02_03:48:23
======================================================================
--------------------------------------------------------------------------------------------------------------------------------
test_name spec board timestamp tguid tguid:result
--------------------------------------------------------------------------------------------------------------------------------
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.add_key04 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.keyctl03 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.keyctl02 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.keyctl05 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.keyctl04 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.keyctl07 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.keyctl06 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.io_getevents01 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.io_submit01 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.fcntl36 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.io_cancel01 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.msgctl11 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.eventfd01 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.request_key02 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.io_setup01 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.io_destroy01 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.add_key03 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.request_key01 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 syscalls.request_key03 FAIL
Functional.LTP default rpi3-1 2018-05-01_01:02:42 sem_init.sem_init_7-1 SKIP
--------------------------------------------------------------------------------------------------------------------------------
}}}
= General Bugs =
- Still having problems with bbb recovering after test
- had a test failure cascade on bbb on 5/1, permission problems with %%(control_dev)s
- ssh: connect to host 192.168.7.2 port 22: Connection timed out
- sh: /dev/serial/by-id/usb-wj@xnk.nu_CDB_Assist_00000042-if02: Permission denied
- Error: Bad result 256, running "echo v >/dev/serial/by-id/usb-wj@xnk.nu_CDB_Assist_00000042-if02": (output follows)
= table of current test status =
If a test is not listed here, it passes on all 4 platforms (bbb, ren1, min1, docker)
{{{#!Table
field=bbb
colors="""(value=="ok"):ccffcc
(value=="fixed"):ffffcc
(value):ffcccc
"""
field=ren1
colors="""(value=="ok"):ccffcc
(value=="fixed"):ffffcc
(value):ffcccc
"""
field=min1
colors="""(value=="ok"):ccffcc
(value=="fixed"):ffffcc
(value):ffcccc
"""
field=rpi3-1
colors="""(value=="ok"):ccffcc
(value=="fixed"):ffffcc
(value):ffcccc
"""
field=docker
colors="""(value=="ok"):ccffcc
(value=="fixed"):ffffcc
(value):ffcccc
"""
||test||bbb||ren1||min1||rpi3-1||docker||notes||disposition||
||Benchmark.OpenSSL||missing zconf.h||.||.||.||.||.||
||Functional.aiostress||missing libaio.o on target||missing libaio.h||missing libaio.so.1 on target||.||.||.||?||
||Functional.crashme||ok||expected 3000, got 507||ok||.||.||.||?||
||Functional.ft2demos||missing libXau and libXmcdp||missing libscb.la||ok||.||.||.||?||
||Functional.glib||processor does not support Thumb mode||program returns 143 exit code||program returns 143 exit code||.||.||.||(high priority)||
||Functional.kernel_build||.||only builds in docker||only builds in docker||.||.||.||needs review||
||Functional.LTP||.||.||.||.||.||default timeout is 40m (too short for some boards, when build is included)||?||
}}}