Contract.. what now?
I have been doing a lot of work with contract testing recently and wanted to do a blog series on something, so this seems as good a thing as anything else! And it’s more technical and I want to do a mix of technical and non-technical blogs.
So, contract testing… some of you may have done this before and some of you may not but it’s becoming more and more relevant given the rise of micro-services and REST APIs.
What is contract testing?
Simply put, contract testing is about assuring the understood and agreed contract between a provider of an endpoint and the consumer(s) of that endpoint.
That sounds simple, and in general it is, but let’s break it down to make it explicit.
The contract
This is the most important part of any contract test and lends itself very nicely to a given/when/then syntax:
Given the provider is in a specific state
When I send the provider http request X with body/headers/query params Y
Then I expect Z to be returned
The given forms a state that you are telling the provider it needs to be in to accept the request. The when forms the details of the request you will send and the then is the response you are expecting, which could take many forms.
The consumer
This is a service, another API, some UI of some kind or anything else that might call the provider’s endpoint. Contract testing is consumer driven meaning it’s down to the consumer to detail its contract and then talk to the provider about that.
The provider
The simplest part really, its responsibility with a contract test is simply to get the contract and ensure that it satisfies it.
What is contract testing useful for?
To explain this well, I need to use some diagrams to explain the gap contract testing covers. Let’s imagine the below is a rudimentary test approach for a small product made up of a REST API providing data to a UI that simply consumes and displays this data:
The provider here is the API and the consumer is the UI. We are ensuring the individual component’s functionality by their own unit and component tests. We also have some kind of end to end automation pack (let’s ignore the details) which assures the business processes for the product.
The question to ask is:
If a provider change breaks the consumer, when do we find out?
We likely won’t find this out through any exploratory testing of the provider and certainly through the provider automation unless we have an encyclopaedic knowledge of the functionality the consumer uses. And that’s just with one consumer…it’s even more impossible with many consumers.
The answer to this question is we might find out when we run the end to end automation, which would require the provider to be running somewhere after we’ve built and deployed it to wherever that is. However if we haven’t covered that case in the automation, we might find it out when we test the app as a whole, or in UAT testing or…gulp…in live.
But, with contract testing.. we find out at build time for the provider.
This is the big thing that contract testing gives you; it can assure that as a provider any change you make will not cause any issues with any of your consumers before even merging your provider code.
What is contract testing not?
Contract testing is for a very specific scenario like I’ve described above, where you have a provider and one or more consumers. It’s even more useful when the consumers make use of different functionality of the provider. There are a few things that contract testing will not give you however that are common misnomers.
Component testing
Contract tests are not there to assure the functionality of either the consumer(s) or the provider. That should be taken care of by the component itself, not by the contracts.
End to end testing
The contract tests do not assure the end to end process through the consumers and the provider, they simply ensure the exit and entry points of each through a contract as described above.
Conclusion
We’ve examined what contract testing is in part 1 and what it can be used for. We’ve also looked at the component parts of the process and a couple of misnomers about it.
In part 2, I’ll introduce the most widely used tool for contract testing called Pact and examine how the Pact framework works.