Auto re run of Failed scenario

sansachiesansachie Posts: 15

i have created a scenario under which there are 3 test cases. I am running this scenario with the option of "tests run as group" and "test run sequentially"
If any of the test case among 3 test cases fail i want re run the complete scenario.
Can anybody help how to achieve this?

Comments

  • tonytony Posts: 8

    You can double-click the parent Scenario, then go to Execution Options: Test Flow Logic: Flow type: While pass/fail. Then choose: Loop until: Every test: Succeeds. Enter the maximum number of loops you want to re-run the scenario in the event that it continues to fail.

    See https://docs.parasoft.com/display/SOA9103/Advanced+Strategies#AdvancedStrategies-LoopingUntilaTestSucceedsorFails-UsingTestFlowLogicLoopingUntilaTestSucceedsorFails-UsingTestFlowLogic

  • sansachiesansachie Posts: 15

    Thanks for answering. But i use SOA 9.8 where i have the option of "Loop until one of the test(s) Succeed/Fails. This doesn't meet my requirement.I want to loop until All the test cases under the scenario are passed.
    Any solution to this would be appreciated.

  • jakubiakjakubiak Posts: 270 ✭✭✭

    Hi @sansachie - the functionality that you are asking for, and that @tony described, was added in version 9.9.3. You will need to upgrade SOAtest in order to get that functionality. The only other option that I can think of is to script your test executions outside of SOAtest from a bash or shell script - where you run the SOAtest test, process the XML report to see if there were failures, and then rerun. But this will be much harder to build and maintain and get the reports you want - upgrading the product would be much easier and would give you what you are looking for. :smile:

  • sansachiesansachie Posts: 15

    Hi @jakubiak - would my testsuits/Scripts be compatible with higher version if i upgarde to higher version of SOA test?

  • jakubiakjakubiak Posts: 270 ✭✭✭

    Yes they will be!

  • sansachiesansachie Posts: 15

    I have upgraded from SOA 9.8 to SOA 9.10 to use re-run feature. Some how it doesnt seem to be working. Below are the logs from console tab. i have test flow logic as in the SS

    ---------- Scenario: Demo ----------
    loop 1 : cycleid variable is 116369
    Test 1: Logic to Pick Test Case - success
    Test 2: randomNumGen - success
    Test 3: addItemCart - failure
    Test 4: addsubItem - skipped
    Test 5: getItemStatus - skipped
    Test 6: getAllItem - skipped
    Loop 2 : cycleid variable is 116369
    Test 1: Logic to Pick Test Case - success
    Test 2: randomNumGen - success
    Test 3: addItemCart - skipped
    Test 4: addsubItem - skipped
    Test 5: getItemStatus - skipped
    Test 6: getAllItem - skipped
    Loop 3 : cycleid variable is 116369
    Test 1: Logic to Pick Test Case - success
    Test 2: randomNumGen - success
    Test 3: addItemCart - skipped
    Test 4: addsubItem - skipped
    Test 5: getItemStatus - skipped
    Test 6: getAllItem - skipped

    Summary - 7 tests ran (85.7% success), 1 failed, 11 skipped

    in loop 1, test 3 has failed but as per the functionality of test flow logic it should re-execute test 3 in Loop 2 but it has skipped it. .

    please refer above SS, our requirement is that if any of the test steps from 1-6 fails, the TC ( or the whole scenario) should be re executed. which is not happening. Please guide .

  • jakubiakjakubiak Posts: 270 ✭✭✭

    I see that Test 3 is not executing during the 2nd and 3rd runs. Is there any other test logic specifically set up for Test 3?

    Based on what you described, the behavior is not what I would expect. However, it's also hard to see whether your scenario has some other configuration that could be causing it to behave this way. The easiest thing would probably be to contact Parasoft Support to have someone debug this with you.

  • sansachiesansachie Posts: 15

    I had TC dependency configured at scenario level and may be this was causing unusual behavior. However without any TC dependency loop feature works fine.

    I was just wondering (it would be great if we have an option) in case any test step under scenario fails why can't we just re try executing the failed Test step alone instead of loop starting from first test step. For reference , from the above Screen shot if Test5:**getItemStatus **fails we want only getItemStatus to be re executed for the number of loops configured instead of starting from Test 1 again. Is that feasible ?

  • jakubiakjakubiak Posts: 270 ✭✭✭

    If that is the behavior that you want, you could put each test into its own test suite, and set the test suite logic in each new test suite to loop until success. We can't make the product do that behavior by default, since some scenarios require the previous steps to be re-run in order for the step that failed to be executed correctly.

Sign In or Register to comment.