{{TableOfContents}}
This page is a dumping ground for ideas for new tests that could be added to Fuego.

Try to keep the tests in categories as shown below.
= Desired system tests overview =
Fuego is a system test framework.

Here are some tests that are good for determining overall system health:
 * LTP system call test (already in Fuego)
 * LTP posix test suite (already in Fuego)
 * LSB-FHS - Linux Standard Base Filesystem Hierarchy Standard test
   * See https://www.opengroup.org/testing/lsb-fhs/

Things to check in a newly created product [edit section]

== Things to check in a newly created product ==
After a change to the kernel or software stack (Linux distribution) on
a product, the following items should be checked:

• interrupts are working (at correct or expected rate) * modules that were expected are loaded correctly * file systems were mounted with correct permissions and attributes * busybox or toybox has all needed sub-commands * shell has needed features (should be part of posix test, but I suspect it's not) * kselftest - for kernel features * libraries that were expected are present

 * interrupts are working (at correct or expected rate)
 * modules that were expected are loaded correctly
 * file systems were mounted with correct permissions and attributes
 * busybox or toybox has all needed sub-commands
 * shell has needed features (should be part of posix test, but I suspect it's not)
 * kselftest - for kernel features
 * libraries that were expected are present

Tests from other distros or projects [edit section]

= Tests from other distros or projects =
Some existing Linux distributions or project have their own selection
of tests, for the package they include.  Here are some candidate
project to get test ideas from (if not the tests themselves).

Apertis tests [edit section]

== Apertis tests ==
See https://gitlab.apertis.org/pkg/apertis-tests.git

AGL tests [edit section]

== AGL tests ==
See https://git.automotivelinux.org/src/qa-testdefinitions/tree/common/scripts

busybox presence test [edit section]

=== busybox presence test ===
see https://git.automotivelinux.org/src/qa-testdefinitions/tree/common/scripts/busybox.sh

service availability test [edit section]

=== service availability test ===
https://git.automotivelinux.org/src/qa-testdefinitions/tree/common/scripts/service-check.sh

CIP tests [edit section]

== CIP tests ==
For overall testing strategy, see https://wiki.linuxfoundation.org/civilinfrastructureplatform/ciptesting/centalisedtesting

Tests are at: https://gitlab.com/cip-project/cip-testing/cip-kernel-tests

Tests are at: https://gitlab.com/cip-project/cip-testing/test-definitions/-/tree/master/automated/linux

CKI (RedHat) tests [edit section]

== CKI (RedHat) tests ==

Debian tests [edit section]

== Debian tests ==

Yocto Project tests (ptests?) [edit section]

== Yocto Project tests (ptests?) ==

Tests by technology area or subsystem [edit section]

= Tests by technology area or subsystem 
== Security tests ==
=== OpenVAS ===
 * check out OpenVAS - a tool that checks for CVEs (from OSSJ)
   * see http://www.openvas.org/

cvecheck [edit section]

=== cvecheck ===
 * see https://github.com/clearlinux/cve-check-tool (last updated 5 years ago)
 * For yocto project, see https://git.yoctoproject.org/poky/tree/meta/classes/cve-check.bbclass

Lynis [edit section]

=== Lynis ===
Lynis is a vulnerability scanner for Linux systems.
 * see https://github.com/CISOfy/lynis

Memory [edit section]

== Memory ==
=== mmtests ===
mmtests by Mel Gorman
 * https://github.com/gormanm/mmtests

filesystem [edit section]

== filesystem ==
=== xfstests ===
xfstests seems to be the new standard for measuring Linux file system
performance.  We should include this test in fuego.

See the following for more information:
 * https://git.kernel.org/cgit/fs/xfs/xfstests-dev.git/
   * clone with 'git clone git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git'
   * I think this is the main upstream repository for xfstests (the repository at http://oss.sgi.com/ has been deprecated)
 * [[https://lwn.net/Articles/592783/|An automated xfstests infrastructure using kvm]]
   * Ted Ts'o's work on automating xfstests
 * [[https://lwn.net/Articles/591985/|Toward better testing]]
   * Dave Chimmer's report on the status of xfstests at an event in 2014

block layer performance measurement [edit section]

== block layer performance measurement ==
Possibly something simple like 'time dd ...' is useful for catching 
some things (and it's short).

Here is a post from Linus Walleij about using a simple dd to measure block layer
performance.  He found a regression of performance using the MQ block layer
scheduler, using this.

I got blk-mq running for MMC/SD today and I see a gross performance
regression, from 37 MB/s to 27 MB/s on Ux500 7.38 GB eMMC
with a simple dd test:

{{{
I got blk-mq running for MMC/SD today and I see a gross performance
regression, from 37 MB/s to 27 MB/s on Ux500 7.38 GB eMMC
with a simple dd test:

BEFORE switching to MQ:

time dd if=/dev/mmcblk3 of=/dev/null bs=1M count=1024
1073741824 bytes (1.0GB) copied, 27.530335 seconds, 37.2MB/s
real    0m 27.54s
user    0m 0.02s
sys     0m 7.56s

AFTER switching to MQ:

time dd if=/dev/mmcblk3 of=/dev/null bs=1M count=1024
1073741824 bytes (1.0GB) copied, 37.170990 seconds, 27.5MB/s
real    0m 37.18s
user    0m 0.02s
sys     0m 7.32s

I will however post my hacky patch as a RFD to the blockdevs and
the block maintainers, along with the numbers and a speculation
about what may be causing it. asynchronous requests (request
pipelining) is one thing, another thing is front/back merge in
the block layer I guess.
}}}

Bus testing [edit section]

== Bus testing ==
=== serial port ===
=== i2c ===
=== USB testing ===
=== CAN bus testing ===
From agl-discussions list Dec 13: https://lists.linuxfoundation.org/pipermail/automotive-discussions/2016-December/003056.html

I'm interested your benchmark amb can data.
http://docs.automotivelinux.org/docs/apis_services/en/dev/reference/iotbzh2016/signaling/AGL-AppFW-CAN-Signaling-Benchmark.pdf

I want to test amb d-bus can data benchmark, 
So can you share your used "can data" and "amd configuration" ?

This test apparently has a CAN packet injector, written by Cogent.

Miscellaneous [edit section]

= Miscellaneous =
== year 2038 test ==
Arnd Bergmann is a leading kernel expert on this topic.  He gave a talk
at Linaro Connect 2017 in Budapest.  See his session at:
http://connect.linaro.org/resource/bud17/bud17-512/
and an lwn.net report on it here: https://lwn.net/Articles/717076/

There's a page with some very small test snippets at:
http://maul.deepsky.com/~merovech/2038.html

list of tests from presentations [edit section]

== list of tests from presentations ==
=== ELCE 2022: linux next testing ===
See https://elinux.org/images/5/50/OSS-EU22-MC-PPT-ELC-linux-next-testing.pdf

From slide 13:
 * modetest, ifconfig, ping, rtcwake
 * inspect /sys/kernel/debug/devices_deferred
 * From Marek's diagram:
   * boot, bash, defer, rtc0, rtc1, fbtest, ping, wifi, bt, alsa, cpu_hp, sleep, sleep2, ping2, reboot, off

ELCE 2022: Growing a lab for upstream testing This talk was about Collabora's KernelCI testing lab. [edit section]

=== ELCE 2022: Growing a lab for upstream testing
This talk was about Collabora's KernelCI testing lab.

See https://elinux.org/images/5/5c/ELCEU2022_Growing_a_Lab_for_Automated_Upstream_Testing.pdf

From slide 27:
 * Test suites:
   * dEQP, Khronos GL and VK CTS, Piglit, trace replaying for OpenGL, Vulkan and
Direct3D, Skqp, va-utils

stress tests [edit section]

= stress tests =
Should probably add stressng
 * http://kernel.ubuntu.com/~cking/stress-ng/

kernel tests [edit section]

= kernel tests =
== kselftest ==

kernelci [edit section]

== kernelci ==
 * build test
 * boot test

device driver tests [edit section]

== device driver tests ==
Texax Instruments has a project (under LTP?) called the "Device Driver Tests"
aka DDT.

See http://processors.wiki.ti.com/index.php/LTP-DDT

Tests from other test frameworks [edit section]

= Tests from other test frameworks =
== 0day ==
Figure out a way to run all existing 0day tests.

(known by Intel as Linux Kernel Performance (LKP) tests)

See https://github.com/intel/lkp-tests

LKFT [edit section]

== LKFT ==
For LKFT's tests, see https://lkft.linaro.org/tests/

kernelci [edit section]

== kernelci ==

syzbot [edit section]

== syzbot ==

CompassCI [edit section]

== CompassCI ==
See https://static.linaro.org/connect/lvc21/presentations/lvc21-202.pdf

Source code at: https://gitee.com/openeuler/compass-ci

Phoronix Test Suite [edit section]

== Phoronix Test Suite ==
See https://openbenchmarking.org/