One problem that is common in working with tests produced outside one's company is that it is difficult to know how to interpret their results. A test user might encounter tests which fail, and which they then have to answer: "what does this test actually do?", "how should its failure be interpreted?", "should this failure stop a release?".
Test users often have to answer these questions themselves, as there is often inadequate documentation about a test or an individual testcase. For example, many individual LTP testcase programs have very limited information available about them.
The 'test documentation' system intends to address this, by providing a way for test users to share information about tests.
The purpose of this is to allow users to create .rst pages for a test, testset, or testcase, and have the system fill in dynamic information on the page, for use in reports or in the user interface (jenkins).
Inside each test directory, there can be a 'docs' directory, where test documentation resides. The files are in 'fuego template' format, which is a custom format similar to .rst. A feugo documentation template file
The page generator is called 'gen-page.py'. It reads an 'ftmp' file, and replaces macros and puts the resulting '.rst' files into /fuego-rw/docs.
The intended data flow is:
- /fuego-core/test/Functional.foo/docs/Functional.foo.ftmp => [gen-page.py] ->
- this replaces Fuego macros with rst syntax for tables or charts
- /fuego-rw/docs/Functional.foo.rst => [sphinx] ->
- this converts rst to html
- when a test is run, it may have a symlink from the log directory for a run to the html file. This will allow the test documentation to appears in the Jenkins interface
For report generation, the results of multiple tests could be combined into a formatted report in a pdf file, such as the following:
This feature is Issue 0088