Auto re run of Failed scenario
i have created a scenario under which there are 3 test cases. I am running this scenario with the option of "tests run as group" and "test run sequentially"
If any of the test case among 3 test cases fail i want re run the complete scenario.
Can anybody help how to achieve this?
Comments
-
You can double-click the parent Scenario, then go to Execution Options: Test Flow Logic: Flow type: While pass/fail. Then choose: Loop until: Every test: Succeeds. Enter the maximum number of loops you want to re-run the scenario in the event that it continues to fail.
1 -
Thanks for answering. But i use SOA 9.8 where i have the option of "Loop until one of the test(s) Succeed/Fails. This doesn't meet my requirement.I want to loop until All the test cases under the scenario are passed.
Any solution to this would be appreciated.0 -
Hi @sansachie - the functionality that you are asking for, and that @tony described, was added in version 9.9.3. You will need to upgrade SOAtest in order to get that functionality. The only other option that I can think of is to script your test executions outside of SOAtest from a bash or shell script - where you run the SOAtest test, process the XML report to see if there were failures, and then rerun. But this will be much harder to build and maintain and get the reports you want - upgrading the product would be much easier and would give you what you are looking for.
1 -
Yes they will be!
0 -
I have upgraded from SOA 9.8 to SOA 9.10 to use re-run feature. Some how it doesnt seem to be working. Below are the logs from console tab. i have test flow logic as in the SS
---------- Scenario: Demo ----------
loop 1 : cycleid variable is 116369
Test 1: Logic to Pick Test Case - success
Test 2: randomNumGen - success
Test 3: addItemCart - failure
Test 4: addsubItem - skipped
Test 5: getItemStatus - skipped
Test 6: getAllItem - skipped
Loop 2 : cycleid variable is 116369
Test 1: Logic to Pick Test Case - success
Test 2: randomNumGen - success
Test 3: addItemCart - skipped
Test 4: addsubItem - skipped
Test 5: getItemStatus - skipped
Test 6: getAllItem - skipped
Loop 3 : cycleid variable is 116369
Test 1: Logic to Pick Test Case - success
Test 2: randomNumGen - success
Test 3: addItemCart - skipped
Test 4: addsubItem - skipped
Test 5: getItemStatus - skipped
Test 6: getAllItem - skippedSummary - 7 tests ran (85.7% success), 1 failed, 11 skipped
in loop 1, test 3 has failed but as per the functionality of test flow logic it should re-execute test 3 in Loop 2 but it has skipped it. .
please refer above SS, our requirement is that if any of the test steps from 1-6 fails, the TC ( or the whole scenario) should be re executed. which is not happening. Please guide .
0 -
I see that Test 3 is not executing during the 2nd and 3rd runs. Is there any other test logic specifically set up for Test 3?
Based on what you described, the behavior is not what I would expect. However, it's also hard to see whether your scenario has some other configuration that could be causing it to behave this way. The easiest thing would probably be to contact Parasoft Support to have someone debug this with you.
0 -
I had TC dependency configured at scenario level and may be this was causing unusual behavior. However without any TC dependency loop feature works fine.
I was just wondering (it would be great if we have an option) in case any test step under scenario fails why can't we just re try executing the failed Test step alone instead of loop starting from first test step. For reference , from the above Screen shot if Test5:**getItemStatus **fails we want only getItemStatus to be re executed for the number of loops configured instead of starting from Test 1 again. Is that feasible ?
0 -
If that is the behavior that you want, you could put each test into its own test suite, and set the test suite logic in each new test suite to loop until success. We can't make the product do that behavior by default, since some scenarios require the previous steps to be re-run in order for the step that failed to be executed correctly.
0
