Pagination - scripting solution or not possible?

goofy78270

I recently came a cross a page that limited the results in the response, however, I am looking to extract all the results to a external file or such.

Sample return in header:
Link: https://sample.com&page=2&per_page=100; rel="next", https://sample.com&page=1&per_page=100; rel="first", https://sample.com&page=24&per_page=100; rel="last"

There is also a previous link added to the list once you go past page 1:
https://sample.com&page=3&per_page=100; rel="prev"

While I can use a Header data bank and search tool to get the next url, I am stumped on how to loop through all pages, without having to code them individually (which would be difficult since page counts can change). I could possibly write to an external CSV or such and read it back every time, but wonder if that is the best solution.

Before I rack my head anymore, I was wondering if anyone else has come across this scenario, and if they possibly have a tested solution they could share.

Thank you in advance for your assistance

jakubiak

What tool are you using to load the URL? Are you using a Browser Testing Tool or a REST Client?

goofy78270

Rest Client

jakubiak

You could create a test suite that contains the REST Client that makes the HTTP request. The REST Client could be parameterized against a Test Suite variable whose initial value is the first URL. The REST Client could have a header data bank chained to it that extracts the value of the next link from the header of the response. The header data bank would write the extracted value into the same test suite variable. You can then configure the test suite to loop until one test fails. When the HTTP response no longer contains the next link (because you have already done through all the data), the REST Client should get an empty URL and should fail - which will break it out of the loop. Since you set it to "loop until fail", the last failure won't actually get reported.

goofy78270

Thanks for the assistance, but I had to go one step further and add an extension tool to parse out the correct url from the header bank variable. Here is the code I created in case others happen to run into a similar issue:

//*************************************************************************
//*************************************************************************
// Import Modules:
//*************************************************************************
//*************************************************************************
import com.parasoft.api.*;
import java.util.*;
import soaptest.api.*;

//*************************************************************************
//*************************************************************************
// Begin Script:
//*************************************************************************
//*************************************************************************
String getLinks(Object input, ExtensionToolContext context)
{
String LinkList = context.getValue("Generated Data Source", "Next URL")
def LinkArray = LinkList.split(',')

def NextURL = '';
for ( L=0; L < LinkArray.size(); L++ ){
    //Application.showMessage("Link " + L + " is " + LinkArray[L].trim())

    if (LinkArray[L].contains('; rel="next"')) {
        //Application.showMessage("Next Link is " + LinkArray[L].trim());

        NextURL = LinkArray[L].substring(LinkArray[L].indexOf('<') + 1, LinkArray[L].indexOf('>'));
        //Application.showMessage("Next URL is actually - " + NextURL);
    }
}
context.setValue("URL", NextURL);

}

On to the next issue...