Benchmark parser notes in 'raw' format
{{TableOfContents}} Here are notes about different elements of the log parser and plot processing. = files = There are several files used: * system-wide (parser and results processing library): * [[common.py]] * fuego_parser_results.py * fuego_prepare_chart_data.py * per-test: * inputs and control files: * [[parser.py]] * [[chart_config.json]] * replaces metrics.json * [[criteria.json]] * replaces reference.log * [[reference.json]] * replaces reference.log * outputs: * results.json * flat_plot_data.txt * flot_chart_data.json * per-run: * run.json = overall flow = The following occurs, during the processing phase of a test: * after running the test, fuego calls [[function_processing|processing]] * this calls parser.py * imports common.py (usually as plib) * calls plib.parse_log with a search pattern * read the test log, and returns a list of matches * puts the results into a python dictionary * calls plib.process() * evaluates the results, according to '''criteria.json''' file * writes the results to '''results.json''' * writes the results to '''flat_plot_data.txt''' * generates chart data * uses '''chart_config.json''' as input * put output in '''flot_chart_data.json''' In the Jenkins interface: * On the status page for the Benchmark test, there are plots automatically generated from the measure data from the test runs for the benchmark. * Jenkins calls the flot (jquery plotter) plugin to show charts for the data based on the flot_chart_data.json file * the code that reads the flot_chart_data.json is in /fuego-core/engine/scripts/mod.js = parser elements = FIXTHIS - These are described elsewhere. == legacy files == These files are deprecated, but are still supported by Fuego: === reference.log === This file has been replaced by '''criteria.json'''. However, if a test does not have a criteria.json file, this file will be read and converted into the necessary structure by the parser library. Each section in the reference.log file has a threshold specifier, which consists of 2 lines: * [metric operator] * threshold value The metric operator line specifies the name of the metric, a vertical bar ('|'), and either 'ge' or 'le', all inside square brackets. The threshold value is a numeric value to compare the benchmark result with, to indicate success or failure. Example: {{{#!YellowBox [SciMark.FFT|ge] 0 [SciMark.LU|ge] 0 }}} See [[reference.log]]