Comparing many fields between API and UI
Data in a distributed system can be transformed and presented in a variety of ways. For example, an API might return data with different cases, leading/trailing whitespace, etc. Also, a UI may present data in an HTML table with additional characters (e.g. nbsp;). Furthermore, if you are trying to validate hundreds of such fields, setting up extractions and assertions that individually compare each of them becomes a test creation & test maintenance nightmare!
In today's post I am going to walk you through how one can compare a dynamic number of data elements between an API's response and an HTML table from a browser' UI efficiently.
The application we will be working with is Parasoft's demo application Parabank. We publically host Parabank here: https://parabank.parasoft.com/ and you can also find the Parabank sources and useful reference automation scripts on our GitHub account here: https://github.com/parasoft/parabank
Let me illustrate the nature of the testing problem we are going to solve today.
Parabank exposes a REST API that returns account details for a particular customer.
When you exercise this API, the JSON response includes an array of account information.
There is no guarantee about how many accounts a customer will have depending on which test environment you're testing in. This is a challenge because your test needs to dynamically handle however many accounts the API returns and compare that data with however many rows exist in the HTML table. The last thing you want to do is spend a lot of time creating individual extractions of all the data elements from the API and then creating individual assertions of each cell in the HTML table only to have this test fail because the test data in your QA environment changed.
Let's handle these dynamic lists in a better way!
The first step to compare dynamic lists of data is to extract all of the important fields from the API response into SOAtest's Writable Data Source. The Writable Data Source exists exactly for these type of scenarios. In my example, I am interested in extracting the accountId and balance from each account that customer '12212' has associated with their account.
The Writable Data Source should be configured to use the Append Standard test mode set to "Per write access".
Next, chain a JSON data bank to your REST Client that is exercising Parabank's GET /customers/{customerId}/accounts resource.
When creating extractions with a data bank, you can right-click on the element in the Tree view and choose an extraction that selects all occurrences of the item's field in a list of items.
You can double click the extraction and Evaluate XPath to confirm it is extracting all of the elements in the list like you want:
Now you want this extracted data to fill up your Writable Data Source columns. In the Modify Selected XPath Settings UI, you have that option in the Data Source Column page:
After doing this for both 'id' and 'balance', you can run your REST Client and see the Writable Data Source populated with data:
The next step is to configure your Scenario with Browser Playback Tools to access the Account Overview page from the browser. This page is visible as soon as you login to Parabank (credentials: john/demo):
The trick here is that you don't want to loop through your Writable Data Source for every browser action, like logging in. You want to open the browser once, login, and then loop through only the Accounts Overview page for each row present on the table. This necessitates adding an additional Browser Playback Tool as your "container" for looping, but you can set the User Action of this tool to be "Wait 0ms". To organize the looping properly, you will also want this Browser Playback Tool to be contained within a Test Suite. See:
For better reporting, you should also open the Test Suite that your container Browser Playback Tool is in and change the Execution Options > Test Execution > Test Relationship to be "Tests run as group".
Ok, now you have all the fundamental building blocks in place to perform the looping validation. In the Browser Playback Tool that does nothing but wait 0ms, add a Browser Validation Tool to the Browser Contents output:
The Browser Validation Tool is what we will be iterating over the Writable Data Source, so you want to double-check that this Browser Playback Tool and chained Browser Validation Tool are both pointing to your Writable Data Source (this may not be set correctly if you already have other data sources in your test, so it's important to check).
Having done that check, now it's time to configure both the Element Locator and Expected Value settings of your Browser Validation Tool. Let's look at the accountId validation first.
The expected value is easy, the data is exactly the same between the JSON response from the API and what is shown in the UI:
The Element Locator requires a little love. Remember, we want this validation to loop through all of the rows on the table. So we need our XPath to be parameterized. The SOAtest Scripting API has a convenient method that returns the data source row index of the current iteration it is on. This numeric index is useful because we can use it to dynamically go down the rows of the table, one-by-one, and then compare the data in the UI versus what the expected data is from the API response that was pushed into the Writable Data Source. I've written a one-line Groovy script that accomplishes this:
import com.parasoft.api.*
public String calculateXPath(input,context) {
return "//TR["+String.valueOf(context.getDataSourceRowIndex())+"]/TD[1]/A"
}
Next, let's look at the balance comparison.
Here, we're using the same Element Locator strategy as before, just choosing the balance column in the HTML table:
However, we have a problem with the expected value. The API returns balances without a $ character, but the UI shows the balance with a $ character. This can be a common problem when comparing data, and depending on what's different the solution will change.
Let's take a look at a simple Groovy script I wrote that will transform the data extracted from the UI to match the syntax of the data coming from the API (don't forget to check the "Use data source" checkbox):
import com.parasoft.api.*
public Boolean compareBalance(input,context) {
String actualValue = context.getActualValue().replace("\$","");
String expectedValue = context.getValue("Bank Data API", "Bank Data API: API_Balance");
if(!actualValue.equals(expectedValue)) {
return false;
}
else {
return true;
}
}
This script is using another SOAtest Scripting API method context.getActualValue() to access the value found by your Browser Validation Tool's Element Locator within your script.
Hopefully this helps you design better test cases that save you time when dealing with dynamic lists of data that need to be compared! You can take this same tactic and apply it to other use cases as well. For example, comparing dynamic lists of data between an API response and ResultSet response from a DB Tool.
I'm attaching the TST file I built during this write-up for your reference. It was created using SOAtest 2021.1 so you will need to be on the latest version as of this posting to open it. Just remove the .txt file extension so the file will be read as SOAtest's .tst extension.