Release test results
1.6-new-container release test results data [edit section]
Key:- (x) = phase (p=pre_check,b=build,r=run)
- LTP prefix: E=Error, F=TFAIL
| test ^ | bbb ^ | min1 ^ | ren1 ^ | rpi3-2 ^ | fuego-test ^ | notes ^ | disposition ^ |
| Benchmark.GLMark | (b) missing glew.h | (b) missing glew.h | (b) missing SDL.h | (b) missing glew.h | . | not new | ? |
| Benchmark.Java | ok (p) missing java | (p) missing java | ok | ok | . | . | ? |
1.5 release test results data [edit section]
Key:- (x) = phase (p=pre_check,b=build,r=run)
- LTP prefix: E=Error, F=TFAIL
| test ^ | bbb ^ | min1 ^ | ren1 ^ | rpi3-2 ^ | fuego-test ^ | notes ^ | disposition ^ |
| Benchmark.GLMark | (b) missing glew.h | (b) missing glew.h | (b) missing SDL.h | (b) missing glew.h | . | not new | ? |
| Benchmark.Interbench | hangs in Audio test | ok | ok | hangs in Audio test | . | omit threadtest or fix? | ? |
| Benchmark.Java | ok (p) missing java | (p) missing java | ok | ok | . | . | ? |
| Benchmark.OpenSSL | ok | ok | ok | (b) missing zconf.h | . | not new | ? |
| Benchmark.bonnie | ok | ok | ok | (p) parser error | . | . | ? |
| Benchmark.cyclictest | non-RT kernel | non-RT kernel | ok | ok | . | . | ? |
| Benchmark.gtkperf | (b) configure error | (b) configure error | deploy error pixbuf | (b) configure error | . | . | ? |
| Benchmark.signaltest | ok | non-RT kernel | non-RT kernel | non-RT kernel | . | . | ? |
| Benchmark.x11perf | (b) missing X libs | can't open display | (b) configure | (b) missing X libs | . | . | ? |
| Functional.LTP | see below | see below | see below | see below | . | . | see below |
| Functional.OpenSSL | ok | ok | ok | (b) missing zconf.h | . | only one board | ? |
| Functional.aiostress | ok | ok | (b) missing libaio.h | fixed missing libaio.so.1 | . | . | ? |
| Functional.crashme | ok | ok | failures (507/3000) | ok | . | . | ? |
| Functional.fontconfig | (p) missing fc_list | ok | ok | ok | . | should have a board skiplist. or visualization for a test that doesn't apply | ? |
| Functional.ft2demos | (b) missing X libs | ok | (b) missing lib | (b) missing X libs | . | . | ? |
| Functional.fuego_release_test | na | na | na | na | (fuego-test) python3 module error | . | ? |
| Functional.glib | (b) thumb mode | (b) configure - gettext | threadtest stops | (b) thumb mode | . | . | ? |
| Functional.kernel_build | ok | ok | (b) head.S error | fixed (b) missing bzImage | . | . | ? |
| LTP.sem_init.sem_init_7-1 | S | S | S | . | . | . | . |
| LTP.syscalls.abort01 | ok | ok | E - setrlimit | ok | . | . | . |
| LTP.syscalls.add_key02 | ok | ok | F hung->timeout | ok | . | . | . |
| LTP.syscalls.add_key03 | E created keyring for other user | ok | E hung->timeout | E created keyring | . | . | . |
| LTP.syscalls.add_key04 | F segfault | ok | F hung->timeout | ok | . | . | . |
| LTP.syscalls.creat08 | ok | F setgid not set | F setgid not set | ok | . | new | RESEARCH |
| LTP.syscalls.cve-2017-5669 | ok | ok | F mapped VM addr | ok | . | . | . |
| LTP.syscalls.cve-2017-6951 | ok | ok | E hung->timeout | ok | . | . | . |
| LTP.syscalls.eventfd01 | ok | ok | ok | F libaio.so.1 | . | . | . |
| LTP.syscalls.fanotify06 | E segfault | E segfault | E segfault | E segfault | . | . | . |
| LTP.syscalls.fcntl35 | F unpriv init | ok | ok | ok | . | . | ?? |
| LTP.syscalls.fcntl35_64 | F unpriv init | ok | ok | ok | . | . | ?? |
| LTP.syscalls.getrusage03 | F (testcase 5) bad ioctl | ok | ok | E bad ioctl | . | . | . |
| LTP.syscalls.io_cancel01 | ok | ok | ok | E libaio.so.1 | . | . | . |
| LTP.syscalls.io_destroy01 | ok | ok | ok | E libaio.so.1 | . | . | . |
| LTP.syscalls.io_getevents01 | ok | ok | ok | E libaio.so.1 | . | . | . |
| LTP.syscalls.io_setup01 | ok | ok | ok | E libaio.so.1 | . | . | . |
| LTP.syscalls.io_submit01 | ok | ok | ok | E libaio.so.1 | . | . | . |
| LTP.syscalls.keyctl02 | F hung ->timeout | ok | F hung ->timeout | ok | . | . | . |
| LTP.syscalls.keyctl03 | F hung ->timeout | ok | F hung ->timeout | ok | . | . | . |
| LTP.syscalls.keyctl04 | F hung ->timeout | ok | F hung ->timeout | ok | . | . | . |
| LTP.syscalls.keyctl05 | F hung ->timeout | ok | F hung ->timeout | ok | . | . | . |
| LTP.syscalls.keyctl06 | F hung ->timeout | ok | F hung ->timeout | F overran buffer | . | . | . |
| LTP.syscalls.keyctl07 | F hung ->timeout | ok | F hung ->timeout | E segfault | . | . | . |
| LTP.syscalls.keyctl08 | ok | ok | F hung ->timeout | E bad ioctl | . | . | . |
| LTP.syscalls.msgctl08 | ok | ok | E no read of ipcs | ok | . | . | . |
| LTP.syscalls.msgctl09 | ok | ok | E no read of ipcs | ok | . | . | . |
| LTP.syscalls.msgctl11 | F queue mismatch, TFAIL at 220 | F queue mismatch | E no read of ipcs | E hangs, 480 procs | . | . | . |
| LTP.syscalls.msgctr10 | F fork failed | ok | E no read of ipcs | ok | . | . | . |
| LTP.syscalls.open10 | ok | F setgid not set | F setgid not set | ok | . | new | RESEARCH |
| LTP.syscalls.prot_hsymlinks | ok | ok | F no useradd hsym | ok | . | . | . |
| LTP.syscalls.recvmsg01 | ok | ok | ok (transient err) | ok | . | . | . |
| LTP.syscalls.renameat201 | ok | ok | F 2,5 invalid arg | ok | . | . | . |
| LTP.syscalls.renameat202 | ok | ok | ok (transient err) | ok | . | . | . |
| LTP.syscalls.request_key01 | F hung ->timeout | ok | ok (transient err) | ok | . | . | . |
| LTP.syscalls.request_key02 | F hung ->timeout | ok | ok (transient err) | ok | . | . | . |
| LTP.syscalls.request_key03 | F hung ->timeout | ok | F segfault | ok | . | . | . |
| LTP.syscalls.sendmsg01 | ok | ok | E ipconfig loopback | ok | . | . | . |
| LTP.syscalls.setpriority01 | ok | ok | E useradd failure | ok | . | . | . |
| LTP.syscalls.setsockopt02 | ok | ok | F bad tp_sizeof_priv | ok | . | . | . |
| LTP.syscalls.utime01 | ok | ok | ok (transient err) | ok | . | . | . |
| LTP.syscalls.utime02 | ok | ok | ok (transient err) | ok | . | . | . |
1.3 test result data [edit section]
Here is a table of 1.3 test results:
| test ^ | bbb ^ | ren1 ^ | min1 ^ | rpi3-1 ^ | docker ^ | notes ^ | disposition ^ |
| Benchmark.OpenSSL | missing zconf.h | . | . | . | . | . | <empty cell> |
| Functional.LTP | . | . | . | . | . | default timeout is 40m (too short for some boards, when build is included) | ? |
| Functional.aiostress | missing libaio.o on target | missing libaio.h | missing libaio.so.1 on target | . | . | . | ? |
| Functional.crashme | ok | expected 3000, got 507 | ok | . | . | . | ? |
| Functional.ft2demos | missing libXau and libXmcdp | missing libscb.la | ok | . | . | . | ? |
| Functional.glib | processor does not support Thumb mode | program returns 143 exit code | program returns 143 exit code | . | . | . | (high priority) |
| Functional.kernel_build | . | only builds in docker | only builds in docker | . | . | . | needs review |
1.4 release test results data [edit section]
If a test is not listed here, it passes on all 5 platforms (bbb, min1, ren1, docker,rpi3-2)
Key: (x) = error occured in phase 'x' (b=build, d=deploy, r=run, etc.)
| test ^ | bbb ^ | min1 ^ | ren1 ^ | rpi3-2 ^ | docker ^ | notes ^ | disposition ^ |
| Benchmark.GLMark | (b) missing glew.h | (b) missing glew.h | (b) missing SDL.h | (b) missing glew.h | . | not new | DEFER |
| Benchmark.Interbench | hangs in Audio test | ok | ok | hangs in Audio test | . | omit threadtest or fix? | DEFER |
| Benchmark.Java | ok | fixed missing java | ok | ok | . | changed to assert_has_program | FIXED |
| Benchmark.OpenSSL | (b) missing zconf.h | ok | ok | (b) missing zconf.h | . | not new | DEFER |
| Benchmark.bonnie | ok | ok | ok | (p) parser error | . | . | DEFER |
| Benchmark.cyclictest | non-RT kernel | non-RT kernel | ok | ok | . | . | ok |
| Benchmark.gtkperf | (b) configure error | (b) configure error | deploy error pixbuf | (b) configure error | . | . | DEFER |
| Benchmark.signaltest | ok | non-RT kernel | non-RT kernel | non-RT kernel | . | . | ok |
| Benchmark.x11perf | (b) missing X libs | can't open display | (b) configure | (b) missing X libs | . | . | DEFER |
| Functional.LTP | new failures | new failures | see below | see below | . | . | see below |
| Functional.OpenSSL | (b) missing zconf.h | ok | ok | (b) missing zconf.h | . | only one board | DEFER |
| Functional.aiostress | fixed missing libaio.so.1 | fixed missing libaio.so.1 | (b) missing libaio.h | fixed missing libaio.so.1 | . | . | FIXED |
| Functional.crashme | ok | ok | failures (507/3000) | ok | . | . | DEFER |
| Functional.fontconfig | missing fc-list | ok | ok | ok | . | . | FIXED |
| Functional.ft2demos | (b) missing X libs | ok | (b) missing lib | (b) missing X libs | . | . | DEFER |
| Functional.fuego_release_test | . | . | . | . | (fuego-test) python3 module error | . | CANDIDATE |
| Functional.fuego_test_minimal | fixed missing spec | fixed missing spec | fixed missing spec | fixed missing spec | fixed missing spec | fix to ftc | FIXED |
| Functional.glib | (b) thumb mode | threadtest stops | threadtest stops | (b) thumb mode | . | . | DEFER |
| Functional.kernel_build | ok | ok | (b) head.S error | fixed (b) missing bzImage | . | . | FIXED (partial) |
| LTP.sem_init.sem_init_7-1 | S | S | S | . | . | . | . |
| LTP.syscalls.abort01 | ok | ok | E - setrlimit | ok | . | . | . |
| LTP.syscalls.add_key02 | ok | ok | F hung->timeout | ok | . | . | . |
| LTP.syscalls.add_key03 | E created keyring | ok | E hung->timeout | E created keyring | . | . | . |
| LTP.syscalls.add_key04 | F segfault | ok | F hung->timeout | ok | . | . | . |
| LTP.syscalls.creat08 | ok | F setgid not set | F setgid not set | ok | . | new | RESEARCH |
| LTP.syscalls.cve-2017-5669 | ok | ok | F mapped VM addr | ok | . | . | . |
| LTP.syscalls.cve-2017-6951 | ok | ok | E hung->timeout | ok | . | . | . |
| LTP.syscalls.eventfd01 | F libaio.so.1 | F libaio.so.1 | ok | F libaio.so.1 | . | . | . |
| LTP.syscalls.fanotify06 | E segfault | E segfault | E segfault | E segfault | . | . | . |
| LTP.syscalls.fcntl35 | F unpriv init | ok | ok | ok | . | . | ?? |
| LTP.syscalls.fcntl35_64 | F unpriv init | ok | ok | ok | . | . | ?? |
| LTP.syscalls.getrusage03 | F bad ioctl | ok | ok | E bad ioctl | . | . | . |
| LTP.syscalls.io_cancel01 | F libaio.so.1 | F libaio.so.1 | ok | E libaio.so.1 | . | . | . |
| LTP.syscalls.io_destroy01 | F libaio.so.1 | F libaio.so.1 | ok | E libaio.so.1 | . | . | . |
| LTP.syscalls.io_getevents01 | F libaio.so.1 | F libaio.so.1 | ok | E libaio.so.1 | . | . | . |
| LTP.syscalls.io_setup01 | F libaio.so.1 | F libaio.so.1 | ok | E libaio.so.1 | . | . | . |
| LTP.syscalls.io_submit01 | F libaio.so.1 | F libaio.so.1 | ok | E libaio.so.1 | . | . | . |
| LTP.syscalls.keyctl02 | F hung ->timeout | ok | F hung ->timeout | ok | . | . | . |
| LTP.syscalls.keyctl03 | F hung ->timeout | ok | F hung ->timeout | ok | . | . | . |
| LTP.syscalls.keyctl04 | F hung ->timeout | ok | F hung ->timeout | ok | . | . | . |
| LTP.syscalls.keyctl05 | F hung ->timeout | ok | F hung ->timeout | ok | . | . | . |
| LTP.syscalls.keyctl06 | F hung ->timeout | ok | F hung ->timeout | F overran buffer | . | . | . |
| LTP.syscalls.keyctl07 | F hung ->timeout | ok | F hung ->timeout | E segfault | . | . | . |
| LTP.syscalls.keyctl08 | ok | ok | F hung ->timeout | E bad ioctl | . | . | . |
| LTP.syscalls.msgctl08 | ok | ok | E no read of ipcs | ok | . | . | . |
| LTP.syscalls.msgctl09 | ok | ok | E no read of ipcs | ok | . | . | . |
| LTP.syscalls.msgctl11 | F TFAIL at 220 | F queue mismatch | E no read of ipcs | E hangs, 480 procs | . | . | . |
| LTP.syscalls.msgctr10 | F fork failed | ok | E no read of ipcs | ok | . | . | . |
| LTP.syscalls.open10 | ok | F setgid not set | F setgid not set | ok | . | new | RESEARCH |
| LTP.syscalls.prot_hsymlinks | ok | ok | F no useradd hsym | ok | . | . | . |
| LTP.syscalls.recvmsg01 | ok | ok | ok (transient err) | ok | . | . | . |
| LTP.syscalls.renameat201 | ok | ok | F 2,5 invalid arg | ok | . | . | . |
| LTP.syscalls.renameat202 | ok | ok | ok (transient err) | ok | . | . | . |
| LTP.syscalls.request_key01 | F hung ->timeout | ok | ok (transient err) | ok | . | . | . |
| LTP.syscalls.request_key02 | F hung ->timeout | ok | ok (transient err) | ok | . | . | . |
| LTP.syscalls.request_key03 | F hung ->timeout | ok | F segfault | ok | . | . | . |
| LTP.syscalls.sendmsg01 | ok | ok | E ipconfig loopback | ok | . | . | . |
| LTP.syscalls.setpriority01 | ok | ok | E useradd failure | ok | . | . | . |
| LTP.syscalls.setsockopt02 | ok | ok | F bad tp_sizeof_priv | ok | . | . | . |
| LTP.syscalls.utime01 | ok | ok | ok (transient err) | ok | . | . | . |
| LTP.syscalls.utime02 | ok | ok | ok (transient err) | ok | . | . | . |
1.3 test result data [edit section]
| test ^ | bbb ^ | ren1 ^ | min1 ^ | rpi3-1 ^ | docker ^ | notes ^ | disposition ^ |
| Benchmark.OpenSSL | missing zconf.h | . | . | . | . | . | <empty cell> |
| Functional.LTP | . | . | . | . | . | default timeout is 40m (too short for some boards, when build is included) | ? |
| Functional.aiostress | missing libaio.o on target | missing libaio.h | missing libaio.so.1 on target | . | . | . | ? |
| Functional.crashme | ok | expected 3000, got 507 | ok | . | . | . | ? |
| Functional.ft2demos | missing libXau and libXmcdp | missing libscb.la | ok | . | . | . | ? |
| Functional.glib | processor does not support Thumb mode | program returns 143 exit code | program returns 143 exit code | . | . | . | (high priority) |
| Functional.kernel_build | . | only builds in docker | only builds in docker | . | . | . | needs review |