Comparing web services for output equality

The other day I had to figure out a way to compare two different web services to see whether a subset of their output are equal or not. One of these services (let’s call it the legacy service) is capable of emitting XML as a response, while the other one (a completely new service that is intended to be a replacement for the legacy code) uses JSON. Unfortunately, the output of these two are only similar in a conceptual way; their structure differs a lot, they use very different ways of structuring data. I checked out some common frameworks for testing web services, and though all of them were useful for parts of my task, they proved to be hard to use for my complete work. Of course, all these frameworks all all too general purpose, while my problem is, I guess, not that of a common use case. At this time I started experimenting with REST Assured – a framework that finally saved me a lot of time and effort. In the below short post I will describe how easy it was to get my job done in a relatively short time, with this cool framework.

First of all, as I mentioned: the hard part of this task was not the inherent difference between XML and JSON – there are really good tools to overcome this issue. The problem was that the logic by which the two services structured their responses was completely different – same data, way too different representation. To make things just a little bit worse, the new endpoint is communicating in a secure manner, through https.

I’m not going to share the exact details of the two responses, but the below represent a simplified (and modified) version of the outputs. First, the legacy XML output:

<countries>
    <country>
           <country_id>10</country_id>
           <providers>
             <provider>
                <id>1</id>
                <name>a_name</name>
             </provider>
           ...
           </providers> 
   </country>
   <country>
           <country_id>11</country_id>
           <providers>
             <provider>
                <id>1</id>
                <name>a_name</name>
             </provider>
           ...
           </providers> 
   </country>
</countries>

And now, the JSON variant”

{ "providers": [
   {
        "id": 1,
        "name": "a_name",
        "active_countries": [10, 11],
        ...
   },
   {
    ...
   }
   ...
]
}

As you can see why the XML document is country-centric, the JSON response takes a different view and puts providers as top level elements.

As JSON is really easy to parse using Jackson I decided to start with this part and stick to its structure for my model objects (the only data I needed was provider id and country ids for that provider). I was more than happy to see that REST Assured has a way of ignoring https validations, so I didn’t need to create and use self-signed certificates and keystores. This way it was really easy and convenient to call the service, parse and convert the response and have the fields I was interested in (less than 50 lines of code):

// ...
public List<Provider> extractNrOfResults() {
    try {
	    String responseBody = makeQuery().body().asString();
	    Response response = objectMapper().readValue(responseBody, Response.class);
	    return responseToProvidersConverter.convert(response);
	} catch (Exception e) {
	    throw new RequestFailedException(e);
	}
}

private RestAssuredResponseImpl makeQuery() {
	RestAssuredResponseImpl response = (RestAssuredResponseImpl) 
			given()
				.relaxedHTTPSValidation()
				.headers(createHeaders())
			.when()
				.body(REQUEST_BODY)
				.post(SERVICE_URL);
	response.then()
				.statusCode(200);

	return response;
}

private Map<String, String> createHeaders() {
	Map<String, String> headers = new HashMap<>();

	headers.put("Content-Type", "application/json");
	// ... more headers ...
	return headers;
}

(note the relaxedHTTPSValidation() method call in the given() clause). First part was ready, but I still needed to extract the data from the XML and convert the result to the same format as in the first case. At the beginning, I was thinking about using JAXB for parsing and a custom converter to translate the resulting objects to the format I picked earlier.

Luckily enough, I did not need to do that. REST Assured does already have another great feature for efficient data acquisition from XML documents, called XmlPath. One can feed XmlPath with Groovy expressions for efficient data retrieval; this makes it really powerful and easy to understand in the same time. In order to get the data I needed out of the XML document, I had to use this much code only:

private static final String ALL_PROVIDER_IDS_QUERY = "regions.depthFirst().findAll{ it.name() == 'id' }";
private static final String COUNTRY_IDS_BY_PROVIDER_QUERY = "countries.country.findAll { country -> country.providers.provider.any{ it.id == '%d' }}*.country_id";
	
protected List<Provider> extract(RestAssuredResponseImpl response) {
	XmlPath path = new XmlPath(response.asString());

	List<Integer> ids = new ArrayList<>(path.getList(XML_ALL_PROVIDER_IDS_QUERY, Integer.class));

	return ids.stream()
			.map(id -> new Provider(id, getRegionIds(path, id)))
			.collect(Collectors.toList());
}

private List<Integer> getRegionIds(XmlPath path, int providerId) {
	return path.getList(String.format(COUNTRY_IDS_BY_PROVIDER_QUERY, providerId), Integer.class);
}

As I am not a Groovy guru, I needed to ask for help; fortunately, I got it pretty fast here. A quick explanation for the fellow Groovy novices: the first query (ALL_PROVIDER_IDS_QUERY) selects all the provider ids from the whole XML document and returns them in a list. The second one (COUNTRY_IDS_BY_PROVIDER_QUERY) extracts all the country codes for a specific provider. Unbelievable, but even easier than JSON parsig.

The only thing that remained was to write a good equals method inside Provider and I was done. I really think REST Assured is an awesome, worth to use tool; it can save a lot of time and effort. It’s worth giving it a try next time you need to test web services.

Advertisements

Author: tamasgyorfi

Senior software engineer, certified enterprise architect and certified Scrum master. Feel free to connect on Twitter: @tamasgyorfi

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s