Various issues with SOATest, need help
Me and my team are facing various issues with SOATest 9.7 (We can not switch to the latest version right away). Please suggest if any one has faced similar issues and got a sustanable fix for these issues. We had use this tool for long but not had a smooth experience ever . In our case we have to deal with 50K+ API heavy api test cases. What i feel this tool is good if you are using it for limited number of tests. But if test grows in size it fails most of time to mange things.
Active Environment Issue
Sometime you trying to run test on X environment, and in background its running on some other environment say Y. Initially it used to happen while you have just open SOATest, and refresh used to help in order to let it run on desired environment. Now it happens pretty often and it behaves randomly. So most of time we are not sure if its actually running to the right environment and we need to and check manually in between.
Even small test suites are hung during execution
Sometime while you are running test and its keep on showing in progress. The things which will finish in 2-3 minutes are stuck for 10s of minutes and doesn't end themselves. Only option we left with is to kill the java process and start it again. And for sure its not my applications who is taking long, its SOATest which is lost somewhere. Due to various issues we kill java process 10s of time in a day .
Its a nightmare and never worked as expected for us. When you run things in concurrent mode, variable-values gets jumbled between multiple running APIs with in tst file. And you see unexpected results most of times.
We are using tool with 4GB heap still it gets crashed very often. I feel memory management with in tool is very poor. When we try to run suites which takes 3+ hours sequentially it will crash in between for sure.