"Test documentation" is the name of a feature of Fuego that is currently under development.

Problem Statement [edit section]

One problem that is common in working with tests produced outside one's company is that it is difficult to know how to interpret their results. A test user might encounter tests which fail, and which they then have to answer: "what does this test actually do?", "how should its failure be interpreted?", "should this failure stop a release?".

Test users often have to answer these questions themselves, as there is often inadequate documentation about a test or an individual testcase. For example, many individual LTP testcase programs have very limited information available about them.

The 'test documentation' system intends to address this, by providing a way for test users to share information about tests.

Proposed solution [edit section]

Here's how it works.

Template files in the test home directory [edit section]

A template file is placed in the test's home directory. A template file can contain static information as well as dynamic information. The dynamic information is obtained by running ftc and generating results data that is used in the documentation.

The purpose of this is to allow users to create .rst pages for a test, testset, or testcase, and have the system fill in dynamic information on the page, for use in reports or in the user interface (jenkins).

Inside each test directory, there can be a 'docs' directory, where test documentation resides. The files are in 'fuego template' format, which is a custom format similar to .rst. A feugo documentation template file

page generator [edit section]

A page generator is used to replace macros in the template file with results from ftc. This page generator outputs files in .rst (reStructuredText) format, which can then be converted into html or pdf, or combined into larger documents, using Sphinx.

The page generator is called 'gen-page.py'. It reads an 'ftmp' file, and replaces macros and puts the resulting '.rst' files into /fuego-rw/docs.

ftmp macros [edit section]

The ftmp macros have the same format as reStructuredText directives.

data flow [edit section]

The intended data flow is:

• /fuego-core/test/Functional.foo/docs/Functional.foo.ftmp => [gen-page.py] ->
  • this replaces Fuego macros with rst syntax for tables or charts
• /fuego-rw/docs/Functional.foo.rst => [sphinx] ->
  • this converts rst to html
• /fuego-rw/docs/Functional.foo.html
  • when a test is run, it may have a symlink from the log directory for a run to the html file. This will allow the test documentation to appears in the Jenkins interface

For report generation, the results of multiple tests could be combined into a formatted report in a pdf file, such as the following:

• /fuego-rw/docs/batch-run-bbb-testplan_default-21.pdf

Status [edit section]

Currently, gen-page.py only generates stub content. The intent is to call ftc or other routines to create the needed content to replace the requested macros in the file.

This feature is Issue 0088