|
"Test documentation" is the name of a feature of Fuego that
is currently under development.
|
"Test documentation" is the name of a feature of Fuego that
is currently under development.
|
|
|
= Problem Statement =
|
|
One problem that is common in working with tests produced outside
one's company is that it is difficult to know how to interpret their
results. A test user might encounter tests which fail, and which they
then have to answer: "what does this test actually do?", "how should
its failure be interpreted?", "should this failure stop a release?".
|
One problem that is common in working with tests produced outside
one's company is that it is difficult to know how to interpret their
results. A test user might encounter tests which fail, and which they
then have to answer: "what does this test actually do?", "how should
its failure be interpreted?", "should this failure stop a release?".
|
|
Test users often have to answer these questions themselves, as there
is often inadequate documentation about a test or an individual testcase.
For example, many individual LTP testcase programs have
very limited information available about them.
|
Test users often have to answer these questions themselves, as there
is often inadequate documentation about a test or an individual testcase.
For example, many individual LTP testcase programs have
very limited information available about them.
|
|
The 'test documentation' system intends to address this, by providing
a way for test users to share information about tests.
|
The 'test documentation' system intends to address this, by providing
a way for test users to share information about tests.
|
|
|
= Proposed solution =
Here's how it works.
|
|
|
== Template files in the test home directory ==
A template file is placed in the test's home directory.
A template file can contain static information as well as
dynamic information. The dynamic information is obtained by
running ftc and generating results data that
is used in the documentation.
|
|
The purpose of this is to allow users to create .rst pages
for a test, testset, or testcase, and have the system
fill in dynamic information on the page, for use in reports
or in the user interface (jenkins).
|
The purpose of this is to allow users to create .rst pages
for a test, testset, or testcase, and have the system
fill in dynamic information on the page, for use in reports
or in the user interface (jenkins).
|
|
Inside each test directory, there can be a 'docs' directory, where
test documentation resides. The files are in 'fuego template' format,
which is a custom format similar to .rst. A feugo documentation
template file
|
Inside each test directory, there can be a 'docs' directory, where
test documentation resides. The files are in 'fuego template' format,
which is a custom format similar to .rst. A feugo documentation
template file
|
|
|
== page generator ==
A page generator is used to replace macros in the template file with
results from ftc. This page generator outputs files in .rst (reStructuredText)
format, which can then be converted into html or pdf, or combined
into larger documents, using Sphinx.
|
|
The page generator is called 'gen-page.py'. It reads an 'ftmp' file,
and replaces macros and puts the resulting '.rst' files into
/fuego-rw/docs.
|
The page generator is called 'gen-page.py'. It reads an 'ftmp' file,
and replaces macros and puts the resulting '.rst' files into
/fuego-rw/docs.
|
|
|
== ftmp macros ==
The ftmp macros have the same format as reStructuredText directives.
|
|
|
== data flow ==
|
|
The intended data flow is:
* /fuego-core/test/Functional.foo/docs/Functional.foo.ftmp => [gen-page.py] ->
* this replaces Fuego macros with rst syntax for tables or charts
* /fuego-rw/docs/Functional.foo.rst => [sphinx] ->
* this converts rst to html
* /fuego-rw/docs/Functional.foo.html
* when a test is run, it may have a symlink from the log directory for a run to the html file. This will allow the test documentation to appears in the Jenkins interface
|
The intended data flow is:
* /fuego-core/test/Functional.foo/docs/Functional.foo.ftmp => [gen-page.py] ->
* this replaces Fuego macros with rst syntax for tables or charts
* /fuego-rw/docs/Functional.foo.rst => [sphinx] ->
* this converts rst to html
* /fuego-rw/docs/Functional.foo.html
* when a test is run, it may have a symlink from the log directory for a run to the html file. This will allow the test documentation to appears in the Jenkins interface
|
|
For report generation, the results of multiple tests could be combined into
a formatted report in a pdf file, such as the following:
|
For report generation, the results of multiple tests could be combined into
a formatted report in a pdf file, such as the following:
|
- /fuego-rw/docs/batch-run-bbb-testplan_default-21.pdf
|
* /fuego-rw/docs/batch-run-bbb-testplan_default-21.pdf
|
|
|
= Status =
Currently, gen-page.py only generates stub content. The intent
is to call ftc or other routines to create the needed content
to replace the requested macros in the file.
|
|
This feature is Issue 0088
|
This feature is [[Issue 0088]]
|