Release_1.3_testing_-_2018-05-01 

Fuego wiki

Login or create account

Release 1.3 testing - 2018-05-01

Here are some notes on testing the 1.3 release (using the 412-container):

flat problem list [edit section]

Resolution key:
  • FIXED - problem fixed with change in Fuego
  • TRANSIENT - problem went away with subsequent test and no change in Fuego
  • LAB ERROR - problem was with setup, config, or board status, issue can be ignored

Bugs seen:

Here are testplan_lts failures for 2018-05-01

min1 [edit section]

  • min1.default.Functional.aiostress - FAIL - missing shared lib libaio.so.1 (not new)
    • error while loading shared libraries: libaio.so.1: cannot open shared object file: No such file or directory
  • min1.default.Functional.glib - FAIL - exits with error code 143 (not new, high priority)
  • min1.default.Functional.LTP - FAIL - 9 testase failures (see details below)
  • min1.default.Functional.kernel_build - ERROR (expected)
    • can only run on docker
  • min1.default.Benchmark.signaltest - FAILURE (expected - not RT kernel)
    • results did not satisfy threshold
  • min1.default.Benchmark.cyclictest - FAILURE (expected - not RT kernel)
    • results did not satisfy threshold

ren1 [edit section]

  • ren1.default.Functional.aiostress - ERROR missing libaio.h (not new)
    • (build phase): aiostress.c:58:20: fatal error: libaio.h: No such file or directory
  • ren1.default.Fucntional.crashme - FAIL (not new, should examine)
    • results did not satisfy the threshold
      • log_compare "0 ... 3000" (but test result was "0 ... 507")
  • ren1.default.Functional.ft2demos - ERROR - missing lib, can't build ftview?
    • libtool: link: cannot find the library `=/usr/lib/libxcb.la' or unhandled argument `=/usr/lib/libxcb.la' Makefile:470: recipe for target '/fuego-rw/buildzone/ren1.default.Functional.ft2demos-poky-aarch64/ft2demos-2.3.6/bin/ftview' failed
    • make: *** [/fuego-rw/buildzone/ren1.default.Functional.ft2demos-poky-aarch64/ft2demos-2.3.6/bin/ftview] Error 1
    • (NOTE: Also had an error about missing program 'build_error', but that was fixed: /fuego-core/engine/tests/Functional.ft2demos/fuego_test.sh: line 16: build_error: command not found)
  • ren1.default.Functional.glib - FAIL (not new)
    • returned exit code 143
  • ren1.default.Functional.LTP - FAIL - 35 FAILS or SKIPs
    • see details below
  • ren1.default.Functional.kernel_build - ERROR - ABORT
    • can only run on docker

bbb [edit section]

  • bbb.default.Funcational.aiostress - ERROR
  • bbb.default.Functional.LTP - ERROR (critical)
    • LTP test hangs the board!!
      • testcase fanotify07 hangs the board
    • why doesn't the test recover after the hang?
      • ltp-pan should time out (it doesn't)
      • fanotify07 threads should be killed (they aren't)
      • Fuego should reboot the board on post-test (it doesn't)
  • bbb.default.Functional.ft2demos - ERROR
    • /usr/lib/gcc-cross/arm-linux-gnueabihf/6/../../../../arm-linux-gnueabihf/bin/ld: cannot find -lXau
    • /usr/lib/gcc-cross/arm-linux-gnueabihf/6/../../../../arm-linux-gnueabihf/bin/ld: cannot find -lXdmcp
    • collect2: error: ld returned 1 exit status
    • Makefile:426: recipe for target '/fuego-rw/buildzone/bbb.default.Functional.ft2demos-debian-armhf/ft2demos-2.3.6/bin/ftlint' failed
    • make: *** [/fuego-rw/buildzone/bbb.default.Functional.ft2demos-debian-armhf/ft2demos-2.3.6/bin/ftlint] Error 1
  • bbb.default.Functional.glib - ERROR (instructions in Thumb mode)
    • (build phase) /tmp/ccSPOtNr.s:65: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode -
  • bbb.default.Functional.kernel_build - ERROR (expected?)
    • can only run in docker
  • bbb.default.Benchmark.OpenSSL - ERROR (missing zlib zconf.h)
    • (build phase) arm-linux-gnueabihf-gcc -I.. -I../.. -I../asn1 -I../evp -I../../include --sysroot=/ -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -c -o c_zlib.o c_zlib.c
    • In file included from c_zlib.c:25:0:
    • /usr/include/zlib.h:34:19: fatal error: zconf.h: No such file or directory
    • #include "zconf.h"

rpi3-1 [edit section]

  • rpi3-1.default.Functional.aiostress - ERROR
  • rpi3-1.default.Benchmark.ffsb
  • rpi3-1.default.Functional.glib - ERROR
  • rpi3-1.default.Functional.f2demos
  • rpi3-1.default.Functional.LTP
  • rpi3-1.default.Functional.bc - ERROR - no BC program on board

NOTES on test duration [edit section]

  • bbb.default.Functional.linus_stress took 27 minutes on 2018-05-01

Overview [edit section]

  root@tlinux:/# ftc list-runs --where "start_time>yesterday,status!=PASS" | sort -k 3
Runs on this host:
where start_time value is '1.5250788e+12'
  Test                        Started                host:board                     Status
  --------------------------- ---------------------- ------------------------------ ------
  Functional.LTP              2018-05-01_01:18:20    fuegohost:bbb                  ERROR
  Functional.aiostress        2018-05-01_01:55:25    fuegohost:bbb                  ERROR
  Functional.crashme          2018-05-01_02:25:11    fuegohost:bbb                  ERROR
  Functional.ft2demos         2018-05-01_02:43:04    fuegohost:bbb                  ERROR
  Functional.glib             2018-05-01_02:54:57    fuegohost:bbb                  ERROR
  Functional.kernel_build     2018-05-01_00:44:46    fuegohost:bbb                  ERROR
  Functional.linus_stress     2018-05-01_03:24:50    fuegohost:bbb                  ERROR
  Functional.netperf          2018-05-01_03:36:43    fuegohost:bbb                  ERROR
  Functional.rmaptest         2018-05-01_03:48:36    fuegohost:bbb                  ERROR
  Functional.synctest         2018-05-01_04:00:29    fuegohost:bbb                  ERROR
  Functional.LTP              2018-05-01_00:43:58    fuegohost:min1                 FAIL
  Functional.aiostress        2018-05-01_01:34:38    fuegohost:min1                 FAIL
  Functional.glib             2018-05-01_01:37:38    fuegohost:min1                 FAIL
  Functional.kernel_build     2018-05-01_01:39:42    fuegohost:min1                 ERROR
  Functional.LTP              2018-05-01_01:01:23    fuegohost:ren1                 FAIL
  Functional.aiostress        2018-05-01_00:54:12    fuegohost:ren1                 ERROR
  Functional.crashme          2018-05-01_00:56:46    fuegohost:ren1                 FAIL
  Functional.ft2demos         2018-05-01_00:55:07    fuegohost:ren1                 ERROR
  Functional.glib             2018-05-01_00:49:26    fuegohost:ren1                 FAIL
  Functional.kernel_build     2018-05-01_00:47:49    fuegohost:ren1                 ERROR
  Benchmark.ffsb              2018-05-01_00:59:25    fuegohost:rpi3-1               SKIP
  Functional.LTP              2018-05-01_01:02:42    fuegohost:rpi3-1               FAIL
  Functional.aiostress        2018-05-01_00:44:39    fuegohost:rpi3-1               FAIL
  Functional.bc               2018-05-01_02:35:33    fuegohost:rpi3-1               ERROR
  Functional.ft2demos         2018-05-01_00:45:07    fuegohost:rpi3-1               ERROR
  Functional.glib             2018-05-01_02:38:14    fuegohost:rpi3-1               ERROR
  Functional.kernel_build     2018-05-01_01:01:38    fuegohost:rpi3-1               ERROR
  --------------------------- ---------------------- ------------------------------ ------

Details [edit section]

min1 LTP testcase problems [edit section]

root@tlinux:/# ftc gen-report --where "board=min1,test=LTP,start_time>yesterday,tguid:result!=PASS"
where start_time value is '1.5250788e+12'
======================================================================
           **** Fuego Test Report ****
  test                : LTP
  board               : min1
  kernel              : 4.13.0-39-generic
  timestamp           : 2018-05-01_00:43:58
  report_date         : 2018-05-01_17:55:27
======================================================================
--------------------------------------------------------------------------------------------------------------------------------
  test_name            spec                 board                timestamp            tguid                tguid:result
--------------------------------------------------------------------------------------------------------------------------------
  Functional.LTP       default              min1                 2018-05-01_00:43:58  syscalls             FAIL
  Functional.LTP       default              min1                 2018-05-01_00:43:58  syscalls.io_getevents01 FAIL
  Functional.LTP       default              min1                 2018-05-01_00:43:58  syscalls.fanotify06  FAIL
  Functional.LTP       default              min1                 2018-05-01_00:43:58  syscalls.io_submit01 FAIL
  Functional.LTP       default              min1                 2018-05-01_00:43:58  syscalls.io_cancel01 FAIL
  Functional.LTP       default              min1                 2018-05-01_00:43:58  syscalls.msgctl11    FAIL
  Functional.LTP       default              min1                 2018-05-01_00:43:58  syscalls.eventfd01   FAIL
  Functional.LTP       default              min1                 2018-05-01_00:43:58  syscalls.io_setup01  FAIL
  Functional.LTP       default              min1                 2018-05-01_00:43:58  syscalls.io_destroy01 FAIL
  Functional.LTP       default              min1                 2018-05-01_00:43:58  sem_init.sem_init_7-1 SKIP
--------------------------------------------------------------------------------------------------------------------------------

ren1 LTP testcase problems [edit section]

35 fails or skips

root@tlinux:/# ftc gen-report --where "board=ren1,test=LTP,start_time>yesterday,tguid:result!=PASS" --fields timestamp,build_number,kernel,tguid,tguid:result
where start_time value is '1.5250788e+12'
======================================================================
           **** Fuego Test Report ****
  test                : LTP
  board               : ren1
  kernel              : 4.9.0-yocto-standard
  timestamp           : 2018-05-01_01:01:23
  report_date         : 2018-05-01_18:45:22
======================================================================
-----------------------------------------------------------------------------------------------------------
  timestamp            build_number         kernel               tguid                tguid:result
-----------------------------------------------------------------------------------------------------------
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls             FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.cve-2017-6951 FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.prot_hsymlinks FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.renameat202 FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.add_key04   FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.keyctl08    FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.keyctl03    FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.keyctl02    FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.keyctl05    FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.keyctl04    FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.keyctl07    FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.keyctl06    FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.msgctl09    FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.msgctl08    FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.fanotify08  FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.fanotify07  FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.fanotify06  FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.utime02     FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.perf_event_open02 FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.abort01     FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.recvmsg01   FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.kill11      FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.msgctl10    FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.msgctl11    FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.request_key02 FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.open10      FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.renameat201 FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.add_key03   FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.setsockopt02 FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.cve-2017-5669 FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.sendmsg01   FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.request_key01 FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.request_key03 FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.setpriority01 FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard syscalls.utime01     FAIL
  2018-05-01_01:01:23  3                    4.9.0-yocto-standard sem_init.sem_init_7-1 SKIP
-----------------------------------------------------------------------------------------------------------

bbb.default.Functional.glib build error details [edit section]

libtool: compile:  arm-linux-gnueabihf-gcc -DHAVE_CONFIG_H -I. -I.. -I.. \
   -DG_LOG_DOMAIN=\"GLib\" -DG_DISABLE_CAST_CHECKS -DG_DISABLE_DEPRECATED \
   -DGLIB_COMPILATION -DPCRE_STATIC -DG_DISABLE_SINGLE_INCLUDES -pthread \
   -g -O2 -Wall -MT gatomic.lo -MD -MP -MF .deps/gatomic.Tpo -c gatomic.c \
   -fPIC -DPIC -o .libs/gatomic.o
/tmp/ccSPOtNr.s: Assembler messages:
/tmp/ccSPOtNr.s:65: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode
/tmp/ccSPOtNr.s:85: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode
/tmp/ccSPOtNr.s:188: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode
/tmp/ccSPOtNr.s:208: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode
/tmp/ccSPOtNr.s:310: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode
/tmp/ccSPOtNr.s:330: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode
/tmp/ccSPOtNr.s:432: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode
/tmp/ccSPOtNr.s:452: Error: selected processor does not support `swp r3,r5,[r4]' in Thumb mode
Makefile:994: recipe for target 'gatomic.lo' failed

rpi3-11 LTP testcase problems [edit section]

root@tlinux:/# ftc gen-report --where "board=rpi3-1,test=LTP,tguid:result!=PASS,start_time>2 days ago"
where start_time value is '1.525060103e+12'
======================================================================
           **** Fuego Test Report ****
  test                : LTP
  board               : rpi3-1
  kernel              : 4.9.41-v7+
  timestamp           : 2018-05-01_01:02:42
  report_date         : 2018-05-02_03:48:23
======================================================================
--------------------------------------------------------------------------------------------------------------------------------
  test_name            spec                 board                timestamp            tguid                tguid:result
--------------------------------------------------------------------------------------------------------------------------------
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls             FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.add_key04   FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.keyctl03    FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.keyctl02    FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.keyctl05    FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.keyctl04    FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.keyctl07    FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.keyctl06    FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.io_getevents01 FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.io_submit01 FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.fcntl36     FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.io_cancel01 FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.msgctl11    FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.eventfd01   FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.request_key02 FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.io_setup01  FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.io_destroy01 FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.add_key03   FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.request_key01 FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  syscalls.request_key03 FAIL
  Functional.LTP       default              rpi3-1               2018-05-01_01:02:42  sem_init.sem_init_7-1 SKIP
--------------------------------------------------------------------------------------------------------------------------------

General Bugs [edit section]

  • Still having problems with bbb recovering after test
    • had a test failure cascade on bbb on 5/1, permission problems with %(control_dev)s
    • ssh: connect to host 192.168.7.2 port 22: Connection timed out
    • sh: /dev/serial/by-id/usb-wj@xnk.nu_CDB_Assist_00000042-if02: Permission denied
    • Error: Bad result 256, running "echo v >/dev/serial/by-id/usb-wj@xnk.nu_CDB_Assist_00000042-if02": (output follows)

table of current test status [edit section]

If a test is not listed here, it passes on all 4 platforms (bbb, ren1, min1, docker)

test  ^ bbb  ^ ren1  ^ min1  ^ rpi3-1  ^ docker  ^ notes  ^ disposition  ^
Functional.LTP . . . . . default timeout is 40m (too short for some boards, when build is included) ?
Benchmark.OpenSSL missing zconf.h . . . . . <empty cell>
Functional.aiostress missing libaio.o on target missing libaio.h missing libaio.so.1 on target . . . ?
Functional.ft2demos missing libXau and libXmcdp missing libscb.la ok . . . ?
Functional.crashme ok expected 3000, got 507 ok . . . ?
Functional.kernel_build . only builds in docker only builds in docker . . . needs review
Functional.glib processor does not support Thumb mode program returns 143 exit code program returns 143 exit code . . . (high priority)
Query

TBWiki engine 1.8.3 by Tim Bird