Functional.LTP issues

Here is a list of bugs with Functional.LTP:

bugs [edit section]

ltp_process.py can't import Borders (raises python exception)
- this program is supposed to produce a spreadsheet containing test results
- it used to run OK, but doesn't at all now
  - it may have been a problem switching from jessie to stretch
    - did python openpyxl module massively change from jessie to stretch
- first problem was exception with: from openpyxl.style import ...
  - sub-module is names 'styles' (with an 's')
- second problem is that module now can't import 'Borders'
  - I'm not sure how to correct it
    - my guess is the openpyxl changed APIs sometime in the last 3 years
      - See https://stackoverflow.com/questions/24917201/applying-borders-to-a-cell-in-openpyxl for how to work with Borders and Sides
      - also: https://realpython.com/openpyxl-excel-spreadsheets-python/#adding-styles
    - What is now "Side" used to be "Borders"?
when test aborts due to a timeout, there is no log
- should use grab-as-you-go logging instead? (report_live?)
- LTP doesn't use the normal Fuego logging system anyway
  - it gathers stuff into a bunch of separate files on the target, and then grabs them and processes them on the host as a separate operation
    - this doesn't interact with Fuego timeout very well
multiple items on a single line in the skiplist file don't get processed
- see line in runltp that processes skiplist
  - for test_name in $(awk '{print $1}' "${SKIPFILE}"); do

LTP requires awk on the target
- runltp uses awk a LOT, to process skiplist and other lists
ftc gen-report shows tguid:result as None when FAIL is in the run.json file
- ftc gen-report --where
- may have to do with test (scenario file name) including a '.'
  - "net.features" adds an extra period to the tguid
    - tguid is "net.features.dccp01"

skipped tests are considered a PASS
- if test is in a skiplist, it's command is replaced with "exit 32;" in the alltests file passed to ltp-pan
- it's also listed as "termination_id=32" in the output log
- there are no individual testcases to parse in the test output file, so the Fuego parser has nothing to report.

shows issues for all boards on an individual board's job page
- ie bbb results are shown on min1.default.Functional.LTP job page
takes too long to calculate skiplist
can't tell what testcase is executing
- with old system, Jenkins console log is completely silent during a whole scenario file full of tests
- with new system, you only see the machine summary line of the tests that have already completed