Contents: Summary field details SAMPLE Ideas

Summary [edit section]

The run.json file has data about a particular test run. It has information about the test, including the result for the test.

field details [edit section]

• built_on - the name of the board the test was executed on (this is a Jenkins build.xml field, and corresponds to their notion of every test being a "build")
• cause - what triggered the build
• charset - character set for the build information
• description - description of the test
• device - target that the test was executed on
• duration - duration of the test, in milliseconds
• files - a list of files comprising the test results and logs
• host - the Fuego host where the test was run
• keep_log - a Jenkins flag (not sure what it's for)
• num - the run number (corresponds to BUILD_NUM in Jenkins)
• reboot - flag indicating whether the target was rebooted for the test
• rebuild - flag indicating whether the test software should have been rebuilt prior to test execution
• result - the test result as a string. This can be one of:
  • SUCCESS
  • FAILURE
• start_time - start time of the test, in seconds since the epoch
• target - the board the test was executed on
• target_precleanup - flag indicating to clean up the target, prior to the test
• test_name - the name of the test
• testplan - the test plan used
• timestamp - a timestamp for when the test was started - this is used as part of a unique identifier for the run
• workspace - a directory on the host where the test materials were extracted and build, for this test

SAMPLE [edit section]

Here is a sample run.json file, generated with "ftc run-test Functional.hello_world docker testplan_docker"

{
    "built_on": "docker",
    "cause": "unknown",
    "charset": "US-ASCII",
    "description": "Test Functional.hello_world run by ftc",
    "device": "docker",
    "duration": 301,
    "files": [
        "devlog.txt",
        "syslog.before.txt",
        "syslog.after.txt",
        "testlog.txt",
        "consolelog.txt",
        "build.xml"
    ],
    "host": "timdesk",
    "keep_log": "true",
    "num": "5",
    "reboot": "false",
    "rebuild": "false",
    "result": "SUCCESS",
    "start_time": 1490291989976,
    "target": "docker",
    "target_precleanup": "true",
    "test_name": "Functional.hello_world",
    "testplan": "testplan_docker",
    "timestamp": "2017-03-23_17-59-49",
    "workspace": "/fuego-rw/buildzone"
}

Ideas [edit section]

Some changes to the fields might be useful:

• right now, 'cause' is just the Jenkins cause, or "unknown" if the test was executed by ftc.
  • the Jenkins strings are somewhat indecipherable:
    • Here is a Jenkins cause: <hudson.model.Cause_-UserIdCause/><int>1</int>
• we should definitely record the spec used
• built_on should be eliminated (we have host and target), so it's redundant
• result for Benchmarks should be the contents of the res.json file (??)
• it might be worthwhile to add some fields from the board/target:
  • firmware
  • architecture
  • platform
  • transport - not sure about this one
  • distrib
• if we add monitors or side-processes (stressors), then would be good to add information about those as well