Fuego includes a report generator.

Here are some details about this system.

outline [edit section]

= outline =
 * batch jobs can create 'logrun' files for use in generating reports
   * logrun files are in json format and contain results from multiple tests
 * during execution, results are saved to the logrun file
 * a special Jenkins job can create reports (Report.make_pdf)
 * reports.sh has shell functions used to generate reports
 * reports are only created for batch jobs (it appears)
 * external tools are used to generate the log:
   * [[loggen.py]]
     * creates logrun files, and appends data to them
     * also, outputs the login in report xml format
   * [[gentexml.py]]
     * converts logrun file to report xml format
   * texml
     * converts report xml format to report .tex format
   * pdflatex
     * converts report .tex format to pdf

loggen.py and gentexml.py are Fuego programs.  texml and pdflatex
are generic programs for managing documents.

reports generation [edit section]

== reports generation ==
 * PDF reports can be generated from accumulated logrun results, by invoking a special Jenkins job: Reports.make_pdf
   * this job has the command sequence:
{{{#!YellowBox
source /home/jenkins/scripts/reports.sh
gen_report
}}}

See [[function_gen_report|gen_report]]

file formats [edit section]

= file formats =

logrun file format [edit section]

== logrun file format ==
There are two kinds of logrun entries: Functional and Benchmark.

A logrun file is in json format, with top level fields of:

• device * testplan * runLogs - a list of entries of the form: * testName, logFile * testName, testResult * runNumber

 * device
 * testplan
 * runLogs - a list of entries of the form:
   * testName, logFile
   * testName, testResult
 * runNumber

logrun entries [edit section]

=== logrun entries ===
Each logrun entry is a dictionary containing two fields.  For a Benchmark
the second field is the logFile.  For a Functional test, the second field is
the testResult.

• logFile is the filename containing benchmark results (xxx.res.json file) * testResult is a string with the overall result for a functional test * the value comes from the log_compare function and is one of the following: * "test error" * "passed" * "failed"

 * logFile is the filename containing benchmark results (xxx.res.json file)
 * testResult is a string with the overall result for a functional test
    * the value comes from the log_compare function and is one of the following:
      * "test error"
      * "passed"
      * "failed"

res.json file format [edit section]

== res.json file format ==
This file is produced for Benchmarks only.  It is in json format, and contains a single object with a set of pairs

Here is an example res.json for Benchmark.Dhrystone:
{{{#!YellowBox
{
    "Dhrystone": "12500000.0"
}
}}}

Here is the res.json for Benchmark.ffsb. (It's a bit more complicated than most others)
{{{#!YellowBox
{
    "Main.append_fsync.TPercent": "1.393",
    "Main.create_fsync.TPercent": "22.452",
    "Main.delete.OpWeight": "9.302",
    "Main.create_fsync.Transactions": "15477",
    "Syscall_latency.write.Min": "0.000000",
    "Syscall_latency.open.Max": "55.584999",
    "Main.open_close.Transactions": "91",
    "Main.create_fsync.OpWeight": "9.725",
    "Main.stat.TPS": "8.11",
    "Main.delete.TPercent": "0.128",
    "Main.open_close.TPS": "8.03",
    "Syscall_latency.close.Avg": "0.002518",
    "Main.delete.TPS": "7.76",
    "Main.append.TPS": "89.08",
    "Main.append.Transactions": "1010",
    "Main.writeall.OpWeight": "10.888",
    "Main.writeall.Transactions": "13750",
    "Syscall_latency.read.Avg": "0.001175",
    "Main.create.OpWeight": "10.782",
    "Syscall_latency.open.Min": "0.002000",
    "Syscall_latency.unlink.Min": "0.016000",
    "Syscall_latency.write.Avg": "0.002091",
    "Main.writeall_fsync.Transactions": "13976",
    "Main.stat.TPercent": "0.133",
    "Main.stat.OpWeight": "9.725",
    "Main.stat.Transactions": "92",
    "Main.readall.OpWeight": "9.619",
    "Syscall_latency.unlink.Max": "3.539000",
    "Main.writeall.TPS": "1212.78",
    "Main.append.TPercent": "1.465",
    "Syscall_latency.read.Max": "0.440000",
    "Main.create.TPercent": "25.646",
    "Main.create.TPS": "1559.32",
    "Main.create.Transactions": "17679",
    "Syscall_latency.read.Min": "0.000000",
    "Main.readall.Transactions": "5812",
    "Syscall_latency.write.Max": "10.133000",
    "Throughput.Read": "2000.00",
    "Syscall_latency.unlink.Avg": "0.188614",
    "Main.writeall_fsync.OpWeight": "9.514",
    "Main.open_close.OpWeight": "9.619",
    "Syscall_latency.stat.Avg": "0.005761",
    "Throughput.Write": "21700.00",
    "Syscall_latency.open.Avg": "0.172316",
    "Main.writeall_fsync.TPS": "1232.71",
    "Syscall_latency.close.Max": "0.014000",
    "Main.append.OpWeight": "10.677",
    "Syscall_latency.stat.Max": "0.019000",
    "Syscall_latency.close.Min": "0.000000",
    "Main.readall.TPercent": "8.431",
    "Main.readall.TPS": "512.63",
    "Syscall_latency.stat.Min": "0.002000",
    "Main.writeall_fsync.TPercent": "20.274",
    "Main.delete.Transactions": "88",
    "Main.writeall.TPercent": "19.946",
    "Main.create_fsync.TPS": "1365.10",
    "Main.append_fsync.OpWeight": "10.148",
    "Main.open_close.TPercent": "0.132",
    "Main.append_fsync.Transactions": "960",
    "Main.append_fsync.TPS": "84.67"
}
}}}

SEE ALSO [edit section]

== SEE ALSO ==
 * [[loggen.py]], [[gentexml.py]], [[function_check_create_logrun|check_create_logrun]], [[function_check_create_functional_logrun|check_create_functional_logrun]], [[function_set_testres_file|set_testres_file]], [[function_gen_report|gen_report]]