Performance Tuning your Virtualize Server

[Deleted User]

For best performance when using a Virtualize server we recommend the following configurations

Always run your server in "Headless mode"

You can do this utilizing the soavirt.war deployable in a web server like tomcat or launching your Virtualize application with the virtualizecli executable

Example:
<VIRTUALIZE_HOME>virtualizecli.sh -startServer -data <WORKSPACE_LOCATION>

Disable Event Monitoring

Always stop monitoring Virtual Assets—both on the server and on all workstations—before starting a load test.
You can also completely disable monitoring as follows:
1. In the Virtualize Server view, double-click the Local Machine node.
2. Clear “Launch event reporting when server starts” checkbox and save.
3. Stop and re-start the server.

This can also be done from the Continuous Testing Platforms Servers Page for headless deployments

Tomcat configurations

For best performance with Virtualize set the following system properties and make a few tweaks to tomcat’s server.xml.

set some jvm properties to give increased memory (at least 4gb) and to use a different garbage collector. Typically you would set this in the TOMCAT_HOME/bin/setenv file

$JAVA_OPTS=-Xms2048m -Xmx4096m -XX:+UseConcMarkSweepGC -XX:+DisableExplicitGC -XX:+UseCompressedOops -XX:NewRatio=1

We also want to make some changes to the server.xml file in TOMCAT_HOME/conf

Find this connector
<Connector URIEncoding="UTF-8" acceptorThreadCount="2" connectionTimeout="20000" enableLookups="false" maxThreads="750" name="default" port="9080" protocol="HTTP/1.1" redirectPort="9443" server="Parasoft Server"/>

This tells Tomcat to keep a processing thread pool of 750 threads it also sets the acceptor thread count to 2

Also towards the bottom of this same file you will fine the following
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="localhost_access_log." suffix=".txt" pattern="common" resolveHosts="false" /> 

Change this into: 

<!--<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="localhost_access_log." suffix=".txt" pattern="common" resolveHosts="false" /> -->
Basically comment out the valve. This will stop Tomcat from filling up the log location with logging information every time the server is hit

Operating System Configurations:

Windows
If using Windows, there are some parameters you can tune in the Registry:
1. Click Start > run > regedit
2. Browse to HKEY_LOCAL_MACHINE > SYSTEM > CurrentControlSet > services > Tcpip > Parameters and set the MaxUserPort to 7FA6 (32678 decimal)
3. Reduce the TcpTimedWaitDelay to 1E (30 decimal)
4. Restart the machine for changes to take effect
This will increase the number of allowed concurrent TCP connections in Windows (MaxUserPort), and reduce the amount of time that an unused TCP port is reserved before it can be reused (TcpTimedWaitDelay)

Linux
It may be necessary to tune your Linux server for high throughput. The following are some system settings that could be helpful. These should be applied to virtualize and the load generation tool.

ulimit -u 4096

Sets the maximum number of processes available for a single user. Important since java threads are processes, having this number too low could result in the inability to spawn new threads.

sysctl -w net.ipv4.tcp_tw_reuse=1
sysctl -w net.ipv4.tcp_tw_recycle=1

These allow the operating system to quickly reuse and recycle network sockets. This is important while under heavy load as sockets can used and discarded frequently.
sysctl.conf

You may need to tune tcp buffers. Based on your network configuration there are several sysctl parameters that can be modified. Refer to http://fasterdata.es.net/host-tuning/linux/ for more information.

Here is a checklist, taking the above topics in mind that you can refer to when debugging performance issues with Virtualize

1. Turn off performance profiles and Additional delays in your Virtual assets
2. Stop monitoring on all assets
3. Turn off event monitoring and Hit statistics at the server level
4. Verify MaxThreads is set appriopriatly in server.xml (See above).
5. Turn off all logging on the server especially SSL debugging
6. Is this on a specific asset? Are other assets performing like before?
7. If you have any responders, or datasource rows that are unused can they be disabled or removed completely
8. Enable caching on datasources. If not, then we are subject to file system read limitations
9. Check for any scripts that the might be in the asset. If they are disabled does the issue go away?
10. Are here any Network share drives hosting artifacts?
11. Set the proper memory args and garbage collection options
    o "-J-Xms4048m -J-Xmx4096m -J-server -J-XX:+UseConcMarkSweepGC -J-XX:+DisableExplicitGC -J-XX:+UseCompressedOops -J-XX:NewRatio=1"
12. 9.9.5 has both xml and json support for hierarchical parameterization of literal messages "Parameterized Literal" we recommend switching to this view over Form when possible
13. Does this reproduce with the same conditions on another server?

Finally,

Always Separate Virtualize and Data Repository if Possible

If possible, separate Virtualize and Data Repository. This is important for "future proofing" your
deployment. This becomes especially important:
• When Virtualize and Data Repository eventually compete for resources.
• If bad data repository happens to bring down the machine hosting Data Repository. If Virtualize
and Data Repository are separated, this Data Repository failure won’t take the Virtualize
server down with it.

[Deleted User]

Just to add to this:

• If you're very focused on performance, try avoid using proxies whenever possible. You will likely experience better performance if you configure your application to directly invoke your virtual services.
• If you must use a proxy for whatever reason and your proxy and virtual service are located on the same machine, use "127.0.0.1" for the "Service host" field on the outgoing connection in the proxy configuration.

If you instead use the IP address of the local machine or the domain name, request/response time may be greater.

Andrey Madan

Quick question - where is the TOMCAT/bin folder on linux distribution? It should be in the same folder where TOMCAT/conf is - but it's not there in that folder.

[Deleted User]

Its different for newer versions of Tomcat. Check out this article. https://askubuntu.com/questions/135824/what-is-the-tomcat-installation-directory

benken_parasoft

You can also query the package manager for your Linux distro. For example, On Ubuntu you can type something like "apt-file list tomcat8 |grep bin".

[Deleted User]

If you are using the new HTTP listener options for setting multiple ports for Virtualize you will need to configure an additional properties file in the Virtualize workspace to fine tune the performance settings. The details can be found here
https://docs.parasoft.com/display/SOAVIRT9105/HTTP+Configuration#HTTPConfiguration-AdditionalHTTPMessageProxyConfiguration

One interesting item of note here is setting the max worker threads
embedded.connector.maxThreads

Specify the maximum number of request-processing threads the connector should create. This property determines the maximum number of simultaneous requests that can be handled. Default is 200 and you will wan to set this the same as your worker threads in tomcat

Grigori Trofimov

8. Enable caching on datasources. If not, then we are subject to file system read limitations

Is this something that is OK to do when working with stateful data sources? or should we only be enabling caching for 'static' data sources?

williammccusker

I think it is not safe to use caching on a Data Repository Data Sources when using the CRUD tool.