Report_Generator 

Fuego wiki

Login or create account

Report Generator

Fuego includes a report generator.

Here are some details about this system.

outline [edit section]

  • batch jobs can create 'logrun' files for use in generating reports
    • logrun files are in json format and contain results from multiple tests
  • during execution, results are saved to the logrun file
  • a special Jenkins job can create reports (Report.make_pdf)
  • reports.sh has shell functions used to generate reports
  • reports are only created for batch jobs (it appears)
  • external tools are used to generate the log:
    • loggen.py
      • creates logrun files, and appends data to them
      • also, outputs the login in report xml format
    • gentexml.py
      • converts logrun file to report xml format
    • texml
      • converts report xml format to report .tex format
    • pdflatex
      • converts report .tex format to pdf

loggen.py and gentexml.py are Fuego programs. texml and pdflatex are generic programs for managing documents.

reports generation [edit section]

  • PDF reports can be generated from accumulated logrun results, by invoking a special Jenkins job: Reports.make_pdf
    • this job has the command sequence:
        source /home/jenkins/scripts/reports.sh
        gen_report

See gen_report

file formats [edit section]

logrun file format [edit section]

There are two kinds of logrun entries: Functional and Benchmark.

A logrun file is in json format, with top level fields of:

  • device
  • testplan
  • runLogs - a list of entries of the form:
    • testName, logFile
    • testName, testResult
  • runNumber

logrun entries [edit section]

Each logrun entry is a dictionary containing two fields. For a Benchmark the second field is the logFile. For a Functional test, the second field is the testResult.

  • logFile is the filename containing benchmark results (xxx.res.json file)
  • testResult is a string with the overall result for a functional test
    • the value comes from the log_compare function and is one of the following:
      • "test error"
      • "passed"
      • "failed"

res.json file format [edit section]

This file is produced for Benchmarks only. It is in json format, and contains a single object with a set of pairs

Here is an example res.json for Benchmark.Dhrystone:

    {
        "Dhrystone": "12500000.0"
    }

Here is the res.json for Benchmark.ffsb. (It's a bit more complicated than most others)

    {
        "Main.append_fsync.TPercent": "1.393",
        "Main.create_fsync.TPercent": "22.452",
        "Main.delete.OpWeight": "9.302",
        "Main.create_fsync.Transactions": "15477",
        "Syscall_latency.write.Min": "0.000000",
        "Syscall_latency.open.Max": "55.584999",
        "Main.open_close.Transactions": "91",
        "Main.create_fsync.OpWeight": "9.725",
        "Main.stat.TPS": "8.11",
        "Main.delete.TPercent": "0.128",
        "Main.open_close.TPS": "8.03",
        "Syscall_latency.close.Avg": "0.002518",
        "Main.delete.TPS": "7.76",
        "Main.append.TPS": "89.08",
        "Main.append.Transactions": "1010",
        "Main.writeall.OpWeight": "10.888",
        "Main.writeall.Transactions": "13750",
        "Syscall_latency.read.Avg": "0.001175",
        "Main.create.OpWeight": "10.782",
        "Syscall_latency.open.Min": "0.002000",
        "Syscall_latency.unlink.Min": "0.016000",
        "Syscall_latency.write.Avg": "0.002091",
        "Main.writeall_fsync.Transactions": "13976",
        "Main.stat.TPercent": "0.133",
        "Main.stat.OpWeight": "9.725",
        "Main.stat.Transactions": "92",
        "Main.readall.OpWeight": "9.619",
        "Syscall_latency.unlink.Max": "3.539000",
        "Main.writeall.TPS": "1212.78",
        "Main.append.TPercent": "1.465",
        "Syscall_latency.read.Max": "0.440000",
        "Main.create.TPercent": "25.646",
        "Main.create.TPS": "1559.32",
        "Main.create.Transactions": "17679",
        "Syscall_latency.read.Min": "0.000000",
        "Main.readall.Transactions": "5812",
        "Syscall_latency.write.Max": "10.133000",
        "Throughput.Read": "2000.00",
        "Syscall_latency.unlink.Avg": "0.188614",
        "Main.writeall_fsync.OpWeight": "9.514",
        "Main.open_close.OpWeight": "9.619",
        "Syscall_latency.stat.Avg": "0.005761",
        "Throughput.Write": "21700.00",
        "Syscall_latency.open.Avg": "0.172316",
        "Main.writeall_fsync.TPS": "1232.71",
        "Syscall_latency.close.Max": "0.014000",
        "Main.append.OpWeight": "10.677",
        "Syscall_latency.stat.Max": "0.019000",
        "Syscall_latency.close.Min": "0.000000",
        "Main.readall.TPercent": "8.431",
        "Main.readall.TPS": "512.63",
        "Syscall_latency.stat.Min": "0.002000",
        "Main.writeall_fsync.TPercent": "20.274",
        "Main.delete.Transactions": "88",
        "Main.writeall.TPercent": "19.946",
        "Main.create_fsync.TPS": "1365.10",
        "Main.append_fsync.OpWeight": "10.148",
        "Main.open_close.TPercent": "0.132",
        "Main.append_fsync.Transactions": "960",
        "Main.append_fsync.TPS": "84.67"
    }

SEE ALSO [edit section]

TBWiki engine 1.8.2 by Tim Bird