Reviewing test results with known bugs

Reviewing test results can assist you in determining if and when JIRA bug issues on a JIRA server should be closed.

Note:
It is assumed that you are familiar with test results (see Test results for details). Only sections related to known bugs are discussed.

Summary tab

Remember:
The Summary tab is available when the test results are viewed either on TestArchitect Client or a web browser.

Test case summary section: Displays the executed test cases and their test results in terms of passed, failed, passed with warning/error, and passed with known bug.

Table 1. Test case summary

FieldDescription
Total test casts selectedThe number of test cases selected to be executed.
Total test cases executedThe number of test cases that were actually executed.
PassedThe number of passed test cases.
FailedThe number of failed test cases.
Passed with Warning/ErrorThe number of test cases that passed, but at least one warning or error was detected.
Passed with known bugThe number of test cases that failed, but for which each failed check was already marked as a known bug.
Not RunThe number of test cases that were skipped during execution, because the automated test was halted. Note that, when a test case is skipped, its detailed information has the N/a text.
Not FinishedThe number of test cases that were unable to reach the end of an automated run.
Test CaseA list of all executed test cases. Note that each test case name listed in the leftmost column of this list is hyperlinked, so that clicking on it displays the result details on a web browser.
ChecksThe number of checkpoints (check actions) in each test case.
PassedThe number of checkpoints with passed results in each test case.
FailedThe number of checkpoints with ordinary failed results in each test case.
Ignored FailsThe number of failed checkpoints that are marked as known bugs in each test case.
Total FailsThe number of ordinary failed checkpoints plus the number of ignored failed checks in each test case.
WarningThe number of checkpoints producing automation warnings in each test case.
ErrorThe number of checkpoints resulting in automation errors in each test case.

Known Bug Summary tab

Remember:
The contents of Known Bug Summary are organized into a tab when the test results are viewed on a web browser; whereas they are organized into a sub-section when the results are viewed on TestArchitect Client.

This tab displays the information about TA bugs marked in the action lines.

Table 2. Known bug summary

FieldDescription
BugID of the TA bug. Click the Bug ID to view Bug Information in TestArchitect Client.
SummaryA brief one-line summary of the TA bug.
Test CaseThe particular section in the test module in which the TA bug is exposed. The section can be Initial, Final or a specific test case. Click the test case number to view the corresponding test case in the Result Details tab.
StatusThe current status, or life cycle stage, of the marked known bug, as defined in the bug’s Current status field.
This RunThe suggested status of the TA bug for this automation run. (Learn more.)
LineThe line number in the test where the action is marked as a known bug. (Note that any number of action lines may be marked with a given bug ID.) Clicking on the line number allows you to view the corresponding line in the Result Details tab.
Ignored FailsThe number of executions of the marked action line that failed during the run.

Suggested TA bug status

The suggested status, or life cycle stage, of a TA bug is determined based on the combination of the actual status of that bug in a new automation run (Passed, Failed, Passed with Warning/Error, or Passed with known bug) plus its current status. Based on the suggested status, you can decide whether or not the corresponding JIRA bug issue on a JIRA server should be closed.

The following table describes the matrix to determine the suggested bug status for each check action marked with a TA bug.

Current statusSuggested status
Passed resultsFailed resultsPassed with Warning/Error resultsPassed with known bug resultsNot executed results
OpenResolvedOpenWarning/ErrorOpenNot Executed
ClosedClosedReopenedReopened
ReopenedResolvedReopenedReopened
Note:

Copyright © 2024 LogiGear Corporation. All rights reserved. LogiGear is a registered trademark, and Action Based Testing and TestArchitect are trademarks of LogiGear Corporation. All other trademarks contained herein are the property of their respective owners.

LogiGear Corporation

1730 S. Amphlett Blvd. Suite 200, San Mateo, CA 94402

Tel: +1 (650) 572-1400