Megatest

View Ticket
Login
Ticket Hash: d4e2e5ab3742c3ebbc096cdfb9a9fa2b7ffcd5cd
Title: STATE/STATUS logic needs correction. Some transitions are resulting in UNKNOWN
Status: Open Type: Code_Defect
Severity: Severe Priority: Immediate
Subsystem: Resolution: Open
Last Modified: 2013-04-24 17:21:44
Version Found In: v1.54
Description:
main test still says UNKNOWN If test is WARN, top-level should also be WARN ok. So SKIP is working as expected wrt item, but parent level test status is not correct (UNKNOWN)

Ok. So the rollup is not correct Let me check that.

For parent test, I think the conditions should be: All item PASS, parent PASS All item PASS or SKIP, parent PASS one item WARN, rest PASS/SKIP, parent WARN one item FAIL, parent FAIL one item WAIVED, no fail, parent WAIVED precedence is: FAIL, WARN, CHECK, PASS

The following rules need to apply to megatest for status for parent tests with items
1 FAIL item  parent test FAIL no FAIL, 1 CHECK  parent test CHECK no FAIL, no CHECK, 1 WAIVED  parent test WAIVED no FAIL, no CHECK, no WAIVED, 1 WARN  parent test WARN no FAIL, no CHECK, no WAIVED, no WARN, 1 SKIP  parent test PASS all SKIP  parent test SKIP all PASS  parent test PASS
Precedence order is:
FAIL => CHECK => WAIVED => WARN => PASS => SKIP
1 corner case:
If all items SKIP, parent is SKIP. So if total number of ITEMS and total number if SKIP match, then parent is SKIP.
SKIP is a lower priority than PASS as it reflects a preemptive cut of a PASS condition (nothing was actually checked/done) whereas PASS implies actual work to QA.

matt added on 2013-04-24 23:21:44 UTC:

      The following rules need to apply to megatest for status for
      parent tests with items

      1 FAIL item                                      parent test FAIL
      no FAIL, 1 CHECK                                 parent test CHECK
      no FAIL, no CHECK, 1 WAIVED                      parent test WAIVED
      no FAIL, no CHECK, no WAIVED, 1 WARN             parent test WARN
      no FAIL, no CHECK, no WAIVED, no WARN, 1 SKIP    parent test PASS
      all SKIP                                         parent test SKIP
      all PASS                                         parent test PASS

      Precedence order is:

      FAIL => CHECK => WAIVED => WARN => PASS => SKIP

      1 corner case:

      If all items SKIP, parent is SKIP. So if total number of ITEMS and
      total number if SKIP match, then parent is SKIP.

      SKIP is a lower priority than PASS as it reflects a preemptive cut
      of a PASS condition (nothing was actually checked/done) whereas PASS
      implies actual work to QA.