FrontPage 

Fuego wiki

Login or create account

Parser module API in split format

{{TableOfContents}}
The file common.py is the python module for performing benchmark log file processing, and results processing and aggregation.
The file common.py is the python module for performing benchmark log file processing, and results processing and aggregation.
It is used by the parser.py program from the test directory, to process the log after each test run. The data from a test run is processed to: * check numeric values for pass/fail result(by checking against a reference threshold values) * determine the overall result of the test, based on potentially complex results criteria * save the data for use in history and comparison charts
It is used by the parser.py program from the test directory, to process
the log after each test run.  The data from a test run is processed to:
 * check numeric values for pass/fail result(by checking against a reference threshold values)
 * determine the overall result of the test, based on potentially complex results criteria
 * save the data for use in history and comparison charts

Parser API [edit section]

parse_log()
  • parse the data from a test log and return a list of tuples with matching groups from a regular expression * process()
  • process results from a test * split_output_per_testcase() - split testlog into chunks accessible from the Jenkins user interface (one per testcase)
= Parser API =
The following are functions used during log processing, by a test's parser.py program.  
 * [[parser_func_parse_log|parse_log()]] - parse the data from a test log and return a list of tuples with matching groups from a regular expression
 * [[parser_func_process|process()]] - process results from a test
 * [[parser_func_split_output_per_testcase|split_output_per_testcase()]] - split testlog into chunks accessible from the Jenkins user interface (one per testcase)
In general, a parser module will normally call parse_log, then put the data into a dictionay, and call process.
In general, a parser module will normally call '''parse_log''', then
put the data into a dictionay, and call '''process'''.
Please see parser.py for more details and examples of use of the API.
Please see [[parser.py]] for more details and examples of use of the API.

Deprecated API [edit section]

= Deprecated API =
''Note: The following information is for historical purposes only.
Although the API is still present in Fuego, these APIs are deprecated.''
In Fuego version 1.1 and prior, the following functions were used. These are still available for backwards compatibility with tests written for these versions of Fuego. * parse() * process_data()
In Fuego version 1.1 and prior, the following functions were used.
These are still available for backwards compatibility with tests written
for these versions of Fuego.
 * parse()
 * process_data()
(see parser.py for invocation details)
(see [[parser.py]] for invocation details)

parse() [edit section]

== parse() ==
 * input:
    * cur_search_pattern - compiled re search pattern
 * output:
    * list of regular expression matches for each line matching the specified pattern
This routine scans the current log file, using a regular expression. It returns an re match object for each line of the log file that matches the expression.
This routine scans the current log file, using a regular expression.  It 
returns an re match object for each line of the log file that matches the
expression.
This list is used to populate a dictionary of metric/value pairs that can be passed to the process_data function.
This list is used to populate a dictionary of metric/value pairs that can
be passed to the process_data function.

process_data [edit section]

== process_data ==
This is the main routine of the module.  It processes the list of metrics,
and populates various output files for test.
  • input: * ref_section_pat - regular expression used to read reference.log * cur_dict - dictionary of metric/value pairs * m - indicates the size of the plot. It should be one of: 's', 'm', 'l', 'xl' * if 'm', 'l', or 'xl' are used, then a multiplot is created * label - label for the plot
 * input:
   * ref_section_pat - regular expression used to read reference.log
   * cur_dict - dictionary of metric/value pairs
   * m - indicates the size of the plot. It should be one of: 's', 'm', 'l', 'xl'
     * if 'm', 'l', or 'xl' are used, then a multiplot is created
   * label - label for the plot
This routine has the following outline: * write_report_results * read the reference thresholds * check the values against the reference thresholds * store the plot data to a file (plot.data) * create the plot * save the plot to an image file (plot.png)
This routine has the following outline:
 * write_report_results
 * read the reference thresholds
 * check the values against the reference thresholds
 * store the plot data to a file (plot.data)
 * create the plot
 * save the plot to an image file (plot.png)

Developer notes [edit section]

= Developer notes =
== functions in common.py ==
 * hls - print a big warning or error message
 * parse_log(regex_str) - specify a regular expression string to use to parse lines in the log
   * this is a helper function that returns a list of matches (with groups) that the parser.py can use to populate its dictionary of measurements
 * parse(regex_compiled_object)
   * similar to parse_log, but it takes a compiled
regular expression object, and returns a list of matches (with groups)
   * this is deprecated, but left to support legacy tests
 * split_tguid()
 * split_test_id()
 * get_test_case()
 * add_results()
 * init_run_data()
 * get_criterion()
 * check_measure()
 * decide_status()
 * convert_reference_log_to_criteria()
 * load_criteria()
 * apply_criteria()
 * create_default_ref()
 * prepare_run_data()
 * extract_test_case_ids()
 * update_results_json()
 * delete()
 * save_run_json()
 * process(results)
   * results is a dictionary with
     * key=test_case_id (not including measure name)
       * for a functional test, the test_case_id is usually "default.<test_name>"
     * value=list of measures (for a benchmark)
     * or value=string (PASS|FAIL|SKIP) (for a functional test)
 * process_data(ref_sections_pat, test_results, plot_type, label)

call trees [edit section]

== call trees ==
{{{#!YellowBox
 process_data(ref_section_pat, test_results, plot_type, label)
   process_data(measurements)
      prepare_run_data(results)
         run_data = (prepare non-results data structure)
         ref = read reference.json
            or ref = create_default_ref(results)
         init_run_data(run_data, ref)
            (put ref into run_data structure)
            (mark some items as SKIP)
         add_results(results, run_data)
             for each item in results dictionary:
                (check for results type: list or str)
                if list, add measure
                if str, set status for test_case
         apply_criteria(run_data)
             load_criteria()
                (load criteria.json)
                or convert_reference_log_to_criteria()
             check_measure()
                get_criterion()
             decide_status()
                get_criterion()
      save_run_json(run_data)
      update_results_json()
      (return appropriate status)
}}}

miscellaneous notes [edit section]

== miscellaneous notes ==
 * create_default_ref_tim (for docker.hello-fail.Functional.hello_world)
   * ref={'test_sets': [{'test_cases': [{'measurements': [{'status': 'FAIL', 'name': 'Functional'}], 'name': 'default'}], 'name': 'default'}]}
 * create_default_ref
   * ref={'test_sets': [{'test_cases': [{'status': 'FAIL', 'name': 'default'}], 'name': 'default'}]}

data format and tguid rules [edit section]

== data format and tguid rules ==
The current API and the old parser API take different data and allow different
test identifiers.  This sections explains the difference:
Data format for benchmark test with new API * measurements[test_case_id] = [{"name": measure_name, "measure": value}]
Data format for benchmark test with new API
 * measurements[test_case_id] = [{"name": measure_name, "measure": value}]
Data format for benchmark test with old API: * in reference.log * if tguid is a single word, then use that word as the measure name and "default" as the test_case. * e.g. for benchmark.arm, the reference.log has "short". This becomes the fully-qualified tguid: arm.default.arm.short: * test_name = arm, test_case = default, test_case_id = arm, measure = short
Data format for benchmark test with old API:
 * in reference.log
    * if tguid is a single word, then use that word as the  measure name and "default" as the test_case.
      * e.g. for benchmark.arm, the reference.log has "short".  This becomes the fully-qualified tguid: arm.default.arm.short:
        * test_name = arm, test_case = default, test_case_id = arm, measure = short
Data format for functional tests with new API and the old API is the same: * e.g. measurements["status"] = "PASS|FAIL"
Data format for functional tests with new API and the old API is the same:
 * e.g. measurements["status"] = "PASS|FAIL"
TBWiki engine 1.8.2 by Tim Bird