Welcome to the new Parasoft forums! We hope you will enjoy the site and try out some of the new features, like sharing an idea you may have for one of our products or following a category.

Uniform Think Time per Service

lukeliwanag
lukeliwanag Posts: 36 ✭
edited March 2021 in Virtualize

Hi There,

Requesting your assistance on a dilemma i am having as i deal with Performance Testing and our Virtualize is built in such a way that it should mimic PROD-like behaviour especially when it comes to Response times.

I am given a set of Production Average Response time per specific Service and I have updated virtualize's think time under Performance as a fixed value.

During my observation, i notice that Virtualize does send the response at a fixed delay time however, it seems like the response generation itself takes a random amount of time when its under load

For Example:

  • Service 1 has a think time of 600ms
  • Service 1 is called outside of load and has a response time of ~650ms (average of 5 different calls)
  • Service 1 is called during peak of the load and has an RT of ~650ms - ~750ms (average of 5 different calls)

the difference between calls is too high during the load.

Now, i know that i should accommodate for Virtualize Processing time however it seems like it Varies even with same Req / Resp.

I'd like to know how i can make the thinktime more robust than it is. i'd like to know how i could set a 600ms think time and have a 600-610ms on average.

I have seen the Load Dependent Response Delays but that requires me to know TPS per service which, as of right now, i don't have to knowledge how to find.

Could you please assist on this?

Thank you.

Tagged:

Comments

  • williammccusker
    williammccusker Posts: 672 ✭✭✭

    Hi,

    What version of Virtualize are you using? When under load its possible that connections are becoming queued up and have to wait for requests ahead of it to be processed. In that case it can lead to response time growing. In newer versions, say 2020.2.0, there is better handling for performance delays under load that would probably address the issue you are seeing. The other option, if you cannot upgrade, is to increase the number of threads that are being run on the server to a number large enough to support as many concurrent requests as necessary.

  • lukeliwanag
    lukeliwanag Posts: 36 ✭

    Hi William,

    Thank you for your reply. we are currently using version 9.10.1 and unfortunately, might take a while for us to actually get 2020.2.0

    Please see below our current server.xml settings and maybe you can help me tweak it

    <Connector SSLEnabled="true" URIEncoding="UTF-8" acceptCount="150" allowTrace="true" clientAuth="false" connectionTimeout="60000" enableLookups="false" keyAlias="xxx" keystoreFile="xxx" keystorePass="xxx" keystoreType="JKS" maxConnections="15000" maxThreads="750" port="9443" protocol="HTTP/1.1" scheme="https" secure="true" server="Parasoft Server" sslEnabledProtocols="TLSv1,TLSv1.1,TLSv1.2,SSLv2Hello" sslProtocol="TLS" truststoreFile="xxx.jks" truststorePass="xxx" truststoreType="JKS"/>

    These were our settings since we last upgraded our server.

    would it be advisable to tweak this or upgrade to 2020.2.0?

  • williammccusker
    williammccusker Posts: 672 ✭✭✭

    Hi,

    The setting to adjust would be the maxThreads attribute. Do you know how many concurrent requests might be happening underlay? You could guess and maybe try 1000 or 1500 threads.

    Upgrading would be the best option as 2020.2 has much better handling for performance delays that make it unnecessary to configure threads in this way.

  • lukeliwanag
    lukeliwanag Posts: 36 ✭

    Hi William,

    Thank you again for your response. I will try increasing the threads up to 1500 and see if the machine could handle it.

    In terms of concurrent requests, we dont have the a way to grab metrics on Virtualize. but our application is getting 170 TPS when at Peak Load. Virtualize might be having more than that.

    I'll connect with my manager in terms of 2020.2. can you please advise why we do not need to configure server.xml for 2020.2? is server.xml already part of the GUI for this version?

    Thank you

  • williammccusker
    williammccusker Posts: 672 ✭✭✭

    Hi,

    There was an improvement made to how the server would handle performance delays so that they could be processed asynchronously which frees up the worker threads so that they don't become blocked by performance delays. I can't find the release notes for when it was specifically added but the latest release, 2020.2, certainly includes the change.

  • lukeliwanag
    lukeliwanag Posts: 36 ✭

    Thanks a lot for all your help William. i will take this up to my manager and see what he thinks.

    Updating the maxthreads definitely helped with the response times.

  • williammccusker
    williammccusker Posts: 672 ✭✭✭

    Happy to help!

Categories

Tagged