What is contract testing?
If you are a consumer of any API you have to do a lot of integration tests (which can be slow), verify responses of external services and be prepared for regressions.
If you are a provider of an API you want to know if your deploy to production won’t make any of the clients sad… So you write more tests on your side, but sooner or later as your API is evolving you will introduce breaking change in it.
Contract testing makes your life easier, both for consumers and providers. It makes your integration contract tests as fast as unit ones. Thanks to CDC you can precisely show to your provider which part of his API you consume. As a provider you know when your deploy will make a regression before it is deployed to production.
Okay, but what actually is contract testing?
In a few words – it is a paradigm of testing where consumers of an API write use cases to their providers. API owners can run those tests in their own environment, whenever they want, and verify if everything works as expected by them by their clients.
The diagram below shows its common architecture.
I did not mention one important piece of the puzzle – Contracts Broker. It is an application where consumers upload their contracts therefore they are accessible for providers at any time.
For whom?
If you consume a REST API, or if you provide an API to anyone, then you might be interested.
Ok, you are still reading this – What is next? 🙂 The answer is simple – scenarios for Consumers and Providers.
If you are a Consumer:
I would like to show you an evolution from integration test to the contract one. Code below will be written in Java using Pact library (there are versions for nodejs, vanilla JS, Swift and others).
Step 1: Integration Test
public class SimpleIntegrationTest {
@Test
void integrationWithProvider() {
ExternalClient client = new ExternalClient(System.getenv("HOST"), System.getenv("ENDPOINT"));
JsonObject result = client.fetchSomedata(1, "");
Assert.assertNotNull(result);
Assert.assertEquals("someValue", result.getString("key"));
//more assertions
}
}
This approach has some pitfalls:
- It is as fast as your provider – it may slow down your build
- You couple your code with an external service during build phase
- There is no way to automatically notify your provider in case of regression from their side
We can modify it:
Step 2: Introduce Mock Server
public class IntegrationTestWithMockServer {
@Test
void integrationWithProvider() {
ClientAndServer server = ClientAndServer.startClientAndServer(RANDOMIZED_PORT);
HttpRequest request = HttpRequest.request()
.withMethod("GET")
.withPath("/someEndpoint")
.withHeaders(
Header.header("Content-type", "application/json")
)
HttpResponse response = HttpResponse.response()
.withBody("SOME BODY")
.withStatusCode(200);
server.when(request).respond(response);
ExternalClient client = new ExternalClient("HARDCODED_HOST","HARDCODED_ENDPOINT");
JsonObject result = client.fetchSomedata(1, "");
Assert.assertNotNull(result);
Assert.assertEquals("someValue", result.getString("key"));
server.verify(request, VerificationTimes.once());
}
}
What did we achieve?
- We are decoupled from our provider, which means there is no risk that our build will fail when external API is down.
- Our test executes faster.
However, there is one serious issue here – we are not able to verify if actual response has been changed until our application is deployed and crashes on production.
Can we improve it? Yes!
Step 3: Use Pact
Setup buildscript:
plugins {
id "au.com.dius.pact" version "3.5.15"
}
apply plugin: 'application'
def pact_version="3.5.15"
test {
systemProperties['pact.rootDir'] = "$rootDir/Pacts/"
}
pact {
publish {
pactDirectory = "$rootDir/Pacts/"
pactBrokerUrl = "PACT_BROKER_URL"
}
}
dependencies {
testCompile("au.com.dius:pact-jvm-consumer-junit_2.12:$pact_version")
testCompile("au.com.dius:pact-jvm-consumer-java8_2.12:$pact_version")
}
And the test itself:
public class PactTest {
@Test
void pactTest() {
//a minimal expected response from a provider
PactDslJsonBody body = new PactDslJsonBody()
//value of `someField` will be randomised
.booleanType("someField")
.minArrayLike("arrayName", 1, 1)
//value of `nestedField` is expected to be `true`
.booleanType("nestedField", true)
//`anotherNestedField` must be a date in format `yyyy-MM-dd`
.date("anotherNestedField", "yyyy-MM-dd", new Date())
.closeArray()
.asBody();
RequestResponsePact pact = ConsumerPactBuilder
.consumer("THAT'S US")
.hasPactWith("OUR PROVIDER")
.uponReceiving("some description")
.path("/someEndpoint/param/1/param2/abc")
.method("GET")
.headers("Authorization", "Basic 1234")
.willRespondWith()
.status(200)
.body(body)
.toPact();
MockProviderConfig config = MockProviderConfig.createDefault();
PactVerificationResult res = runConsumerTest(pact, config, (mockServer) -> {
ExternalClient client = new ExternalClient(mockServer.getUrl(), "someEndpoint");
JsonObject result = client.fetchSomedata(1, "");
Assert.assertNotNull(result);
Assert.assertEquals("someValue", result.getString("key"));
});
Assert.assertEquals(PactVerificationResult.Ok.INSTANCE, res);
}
}
What did we achieve?
You may say that besides a new DSL… nothing much, but you are not right! Each time you run gradle test, Pact automatically generates JSON file which has your contract inside.
It looks like this:
{
"provider": {
"name": "THAT'S US"
},
"consumer": {
"name": "OUR PROVIDER"
},
"interactions": [
{
"description": "description",
"request": {
"method": "GET",
"path": "/someEndpoint"
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json; charset=UTF-8"
},
"body": {
},
"matchingRules": {
"body": {
"$.result": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
}
}
}
}
}
],
"metadata": {
"pact-specification": {
"version": "3.0.0"
},
"pact-jvm": {
"version": "3.5.15"
}
}
}
When you run gradle pactPublish that contract will be uploaded to the Broker and will be validated each time Provider runs its tests.
Benefits of our changes:
- We are decoupled from provider
- Our tests are fast
- Whenever external API makes a regression in their response it will stop their building process
It sounds good, doesn’t it? That is all that was to be achieved on the Consumer part.
If you are a Provider:
You have almost no work to do – the only thing you need is a working Pact Broker. Then you have to modify your buildscript:
plugins {
id "au.com.dius.pact" version "3.5.15"
}
apply plugin: 'application'
def pact_version="3.5.15"
test {
systemProperties['pact.rootDir'] = "$rootDir/Pacts/"
}
dependencies {
testCompile("au.com.dius:pact-jvm-provider-gradle_2.12:$pact_version")
}
pact {
serviceProviders {
'YOUR API NAME' {
protocol = 'https'
host = System.getenv("DOMAIN")
port = 443
path = '/'
requestFilter = { req ->
//you can modify your request here
}
hasPactsFromPactBroker("YOUR PACT BROKER URL")
}
}
}
When you run gradle pactVerify it will fetch all contracts related to your API and run those against your endpoints. You will see a similar response.
Verifying a pact between X and Y
[from Pact Broker ]
Given sample endpoint
returns a response which
has status code 200 (OK)
includes headers
"Content-Type" with value "application/json" (OK)
has a matching body (OK)
If you make a regression, Pact will inform you about it with the following message:
Failures:
0) Verifying a pact between SAMPLE CONSUMER and YOUR SERVICE - A request for something
Given some endpoint
Reason of failure
You will see a full list of broken contracts and you will know who should be informed about it.
A few words about Pact Broker
As I mentioned above, Pact Broker is a persistence layer for your contracts. It aggregates all interactions between services and draws them as a graph.
What is more, each interaction is presented in a human-readable form.
Pact Broker is an optional element of Contract testing, but thanks to it, you do not have to copy your contracts manually between services. Moreover, all interactions are stored in one place and are updated automatically.
There are some limitations…
- If you provide an API with dynamic authentication (i.e. OAuth tokens) then you have to write custom request filter on your side which will modify headers on the fly
- If your API provides a lot of POST/PUT endpoints you have to take care of providing a proper state of your service for each test (it can be done by state change URL).
…but it is more than enough to make you happy.
To sum up, when you develop a system which is built upon microservices you may find out that Contracts can save a lot of problems you might have.
- From consumer perspective – you can easily show your interactions to a provider. Whenever your business logic changes and you modify the Contract test you automatically notify your provider about it.
- From provider perspective – you do not have to write as many smoke tests as before. You can delegate testing of your API to your clients – all in all, you write it for them, not for yourself. Whenever you make a regression, you will know that before your application is deployed in production environment.