WSDL diff or WSDL Regression versus Diff with XML or Text mode

LegacyForumLegacyForum Posts: 1,669 ✭✭
edited December 2016 in SOAtest
---------- Test 4: WSDL Regression ----------
Test 4: WSDL Regression: 1 test to process...
Did not run XML Diff, there are too many differences. Performing text mode diffing instead.
Test failed

My question is how many difference are too many? Does the diff tool have a different look for the case when there aren't too many differences. I typically use the diff functionality in a pure XML editing product to give me indepth analysis of the differences between WSDLs builds.

Can you force Soa Test to do an XML syntax diff?

Thanks in advance,



  • LegacyForumLegacyForum Posts: 1,669 ✭✭
    Hi LeapTester,

    Please take a look at the following forum post and let me know if you have any questions -
  • LegacyForumLegacyForum Posts: 1,669 ✭✭
    Thanks J for the tip. I'm an idiot for not searching the forum first. thanks for not flaming me

    However i'm running WSDL diffs from the basic WS-I testing profile provided in Soa Test. So diff'ing the entire WSDL makes using Ignore Xpaths a bit cumbersome as these WSDLs can change every day. clicking update regression control vs havin to update individual Xpaths is a much more tenable solution.

    Thanks and i'll play around with those values. Any word on when the partial XML-diff joined with text mode diff feature will be added?

  • LegacyForumLegacyForum Posts: 1,669 ✭✭
    Currently we do not have a timeframe for when this PR will be implemented. If you don't have success using the method which was described in the referenced post, one suggestion for now is to do a plain text diff on the WSDL file. To do this, you can change the Diff mode within the diff tool from "XML" to "Text". You won't get the nice XML diff results but you will still be alerted to any differences that show up.
  • LegacyForumLegacyForum Posts: 1,669 ✭✭
    Here's a stupid question. What are the differences between the XML diff and Text mode diff? Is there any additional logic involved? Graphical output differences?

  • LegacyForumLegacyForum Posts: 1,669 ✭✭
    Well, when it comes to doing a straight Text Diff, SOAtest just performs a simple character by character textual comparison. The XML diff is a bit more advanced. The XML diff compares the instance document element by element and attribute by attribute, ignoring whitespace characters. The result of the comparison is an XML document. You also have the ability to do things like ignoring certain portions of the instance document as well (Ignoring an XPath). And in an XML diff, you are given the XPath of the element/attribute where the differences were found so that you can easily spot the exact location of the change.
  • LegacyForumLegacyForum Posts: 1,669 ✭✭
    Hmmm. Ok J, thanks for the response. I'll keep playing around with the buffer sizes with the method control until I get a value which can contain a WSDL sized diff. Its weird but my WSDLs avg 100K in size. I would think that would be nothing to parse and report?

    Thanks again,

  • [Deleted User][Deleted User] Posts: 0 admin

    Referenced Discussion is inserted here:

    Did not run XML Diff, too many differences

    Options V

    post Nov 3 2005, 02:20 PM
    Post #1


    I am getting this warning message for a fairly large SOAP response. Is there anyway to force it to using the XML Diff?

    ?Did not run XML Diff, there are too many differences. Performing text mode diffing instead.?

    My response has a lot of @id attributes and they change each time. Because of this I want it to do the XML Diff so I can then tell it to add the XPath to the Ignored Differences.


    post Nov 3 2005, 04:31 PM
    Post #2

    XML Diffing is a computationally heavy process and we have to impose limits on the size/complexity of the messages so as not to overwhelm less powerful machines. You can modify these limits by using the following script. If you are not already familiar with scripting in SOAtest, you will need to add a Method tool to your test suite and then add the following Python code to it:

    from org.vmguys.vmtools.utils import DifferenceFinder2
    def increaseDiffTreeSize(x):
    DifferenceFinder2.MAX_SIZE = 62500
    DifferenceFinder2.MAX_SIZE_AFTER_ELIMINATE = 16384

    The values included with the script are the default limits. Try experimenting with larger values and see if you are able to diff the message. Note that if you set these values too high, and the message is too complex, you'll get an OutOfMemory error.

    Let me know if this helps or if you have any other questions.
    Go to the top of the pageReport Post


    post Nov 4 2005, 08:07 PM
    Post #3

    I tried bumping the numbers and now I get the message "Did not run XML Diff, it did not finish within 90.0 secs. Performing text mode diffing instead."

    I would suggest that it should at least list the XML differences it has found in front of the text differences. This would allow me to to narrow them down.


    post Nov 7 2005, 03:50 PM
    Post #4

    How big is your SOAP message?

    Another approach you could take involves the XML Transformer tool. You say you want to ignore a large part of your message, how many values would you actually like to diff? Have you tried using the XML Transformer tool? It uses XPaths to extract specific values in your message. After extracting these values, you can chain a Diff tool to the output and only the values extracted from the XML Transformer tool will be used in the diff. Let me know if you have any questions about setting this up.

    Best Regards,

    • Graeham

    post Nov 8 2005, 05:47 PM
    Post #5

    The message is only 156K. All my external XML diff tools have no problems with it.

    Anyway, I did a variation of your idea and added a XSLT tool to strip away the @id attributes and then created the Diff on that. That seems to do the trick (thanks).

    Still, since I am still using the Diff tool I am wondering why it fails in the original case (right on the SOAP Client). I am 95% sure I gave it a complete list of the XPaths to ignore, so there shouldn't be that big a difference.

    Once again I would suggest that when the XML Diff tool fails out (due to size or time) it should list the XML diffs it has found and then proceed with the text diffs. This would allow the user to add them to ignore list.

    Thanks again,

    post Nov 8 2005, 05:51 PM
    Post #6

    Let me add that the XML Diff, "generate regression test", and automatic "Ignore XPath" functionality is very cool and useful - which is why I am pushing on it so much wink.gif.


    post Nov 9 2005, 03:55 PM
    Post #7

    Hi D,

    Based on your suggestion of having the partial XML diff results output along with the Text diff results, I have filed PR 48534. It will be implemented into a future version of SOAtest.

    Thank you for your suggestion.

    Best Regards,

    post Apr 7 2006, 05:39 AM
    Post #8

    I have used the following suite to run the WSDL Regression Test(Diff in XML mode) between the exisiting wsdl and older version WSDL file.

    Test Suite for Diff

    • Method Tool
    • WSDL Regression Test

    =>For method tool I added the following python script as you mentioned in the above replies.
    from org.vmguys.vmtools.utils import DifferenceFinder2
    def increaseDiffTreeSize(x):
    DifferenceFinder2.MAX_SIZE = 9999900
    DifferenceFinder2.MAX_SIZE_AFTER_ELIMINATE = 999999

    =>I have selected the older WSDL file (in .xml format) as file to "HTTP Response->Diff WSDL" tool which is under "WSDL Regression Test". Why I selected as file is, I don't want to compare current wsdl with just previous one and I want to compare it with much older one.

    Test (suite) is executed in XML Mode. It shows some error like below.

    ERROR1: Changed attribute location value from expected "http://localhost:6060/cmdb_wsdm" to "http://win_rdops2:80/systinet/server/cmdb_wsdm" in XPath /definitions/service/port/address

    ERROR2: Changed attribute message value from expected "tns:CreateResourceResponse" to "wsrf-rpw:UpdateResourcePropertiesResponse" in XPath /definitions/portType/operation[12]/output

    My Questions are:
    Can I get the correct results, by comparing with external xml file like above?
    Am I using the Method tool in right manner?
    How to interpret the above error messages?
    What is XPath shown in error message?

    Thanks in advance smile.gif


    post Apr 10 2006, 03:01 PM
    Post #9

    1) You can use the external file to do the diff. It would be the same as opening the external file and copy/pasting into the diff text box.

    2) There are several ways you could use the Method tool:

    a) Your current setup will work if you run that test suite in sequential mode. In this case, the Method tool and the WSDL Regression tool are the same level so the Method tool will run first and its effects will apply to the WSDL Regression test after it.

    cool.gif You could also add a Set-Up test to add the Method tool to the test suite. Set-Up tests are run before any other test in the test suite in both sequential and concurrent modes.

    c) You could chain the Method tool to the request part of the test to make it run before the response is received. For instance, doing Add Output->HTTP Request->New Output->Method tool, would make the Method tool run before the response.

    d) You could also add a Global Method tool to the test suite. This would add the script to the Existing Output list. Then you could easily add the script to tests instead of copying/pasting many times.

    3) The XPath is showing you what node has the difference. For instance, in the first error message the address node has an attribute that changed value. The older file contained "http://localhost:6060/cmdb_wsdm" and the newer file contained "http://win_rdops2:80/systinet/server/cmdb_wsdm".

    4) XPath is a W3C standard for navigating through an XML document. It uses a syntax similar to file systems. For instance, /definitions/portType/operation[12]/output means that definitions is parent of portType, portType is parent of operation, and operation is parent of output. The [12] means it's the 12th operation node under the portType node.

Sign In or Register to comment.