Consumer-Driven Contract Testing

Context

Microservices are being adopted. The internals of the microservices are hidden behind its API. There is a need to make changes to a microservice API.

Problem

There is uncertainty wheter the API change would break consumers
- Preemptive API versioning leads to unnecessary complexity and effort.
Traditional integration testing to ensure compatibility requires coordination between teams and is expensive

Solution

Use consumer-driven contract tests instead of traditional integration tests between microservices to test for syntactic interoperability.

The API-consuming microservice team expresses its expectations towards the API in a so called consumer contract. This formulated contract can be used by the API-providing microservice to test their changes towards breaking changes. The sum of all consumer contracts towards the same API provider - called consumer-driven contract - serves as a safety net for changes on the API contracts.

CDCT Sketch

Consumer-driven contract tests can serve as documentation to see dependencies between microservices and the used endpoints. As they can be applied to every API version they can also serve as documentations of version compatibility.

Using consumer-driven contract testing contributes to reducing coordination overhead between microservice teams as the contract is expressed very explicitly. Testing becomes more detached as only the API consumer and API provider are involved to test a consumer contract. The test execution of both sides runs in isolation, so there is no need start several microservices. Summarizing, it contributes to the independence of microservices and their responsible teams.

Adding this new kind of testing might increase the complexity of the testing strategy. Mocking and stubbing might be a challenge that needs to be overcome first to reap the benefits of this best practice.

Example

This example is adapted from the example on https://github.com/pact-foundation/pact-net and uses a C# library for consumer-driven contract testing called Pact.

API Consumer Side

public class PatientApiConsumerTests
{

    public PatientApiConsumerTests(ITestOutputHelper output)
    {
        // Setup pact library
        // ...
    }

    [Fact]
    public async Task GetPatient_WhenTheTesterPatientExists_ReturnsThePatient()
    {
        // Arrange
        _pactBuilder
            .UponReceiving("A GET request to retrieve the patient")
                .Given("There is a patient with id 'tester'")
                .WithRequest(HttpMethod.Get, "/patients/tester")
                .WithHeader("Accept", "application/json")
            .WillRespond()
                .WithStatus(HttpStatusCode.OK)
                .WithHeader("Content-Type", "application/json; charset=utf-8")
                .WithJsonBody(new
                {
                    id = "tester",
                    firstName = "John",
                    lastName = "Doe"
                });

        await _pactBuilder.VerifyAsync(async ctx =>
        {
            // Act
            var client = new PatientApiClient(ctx.MockServerUri);
            var patient = await client.GetPatient("tester");

            // Assert
            Assert.Equal("tester", patient.Id);
        });
    }
}

The sequence diagram only roughly reflects how the pact ecosystem works and might differ from implementation details of Pact.

Output: Pact File containing consumer interactions and the expected responses - a single consumer contract. This Pact File is passed to the provider tests to verify if the consumer expectations are in line with the implementation of the provider.

API Provider Side

public class PatientApiTests : IClassFixture<PatientApiFixture>
{
    private readonly PatientApiFixture fixture;
    private readonly ITestOutputHelper output;

    public PatientApiTests(PatientApiFixture fixture, ITestOutputHelper output)
    {
        this.fixture = fixture;
        this.output = output;
    }

    [Fact]
    public void EnsurePatientApiHonoursPactWithConsumer()
    {
        //Arrange
        var config = new PactVerifierConfig
        {
            // Configure pact
            // ...
        };

        string pactPath = Path.Combine("..", "..","path", "to", "pacts", "Patient API Consumer-Patient API.json");

        // Act / Assert
        IPactVerifier pactVerifier = new PactVerifier(config);
        pactVerifier
            .ServiceProvider("Patient API", fixture.ServerUri)
            .WithFileSource(new FileInfo(pactPath))
            .WithProviderStateUrl(new Uri(fixture.ServerUri, "/provider-states"))
            .Verify();
    }
}

The sequence diagram only roughly reflects how the pact ecosystem works and might differ from implementation details of Pact.

Maturity

Proposed, evaluation required.

Sources of Evidence

L3:

replaced integration tests with cdc tests
- => independent testing of each service
- uses consumer's expectations
- (+) minimized interteam coordination
  - => enabled forming smaller teams
  - => independent testing enabled to deploy services independently
- (-) more complex testing strategy
consumer-driven contracts could increase confidence in the team's responsible for service
- most of its customers are satisfied if contracts olds

L12:

same as gs3

L14:

Context: additional concerns at otto.de not covered in paper
- customer-driven contracts (among others)

L22:

Consumer-driven contracts as complementary practice to microservices

L32:

devs of consuming service write contract tests
- to ensure producing services meet their expectations
contract tests also run in pipeline of producing services
- => devs of producing services know whether changes will break expected contracts of consumers
=> safety net for contract changes

L54:

Cites consumer-driven contracts as alternative to service versioning to solve cross-configuration
- cross-configuration = config change on service (e.g. IP) impacts other services

Interview A:

Contracts when a new service consumption is implemented
- contract over their REST interface
- Tools like Swagger
- Contract testing as one of the upcoming topics

Interview B:

Where happens testing?
Theoretically, if I test APIs diligently
- e.g. automated CDC tests
- and communicate every contract change
- and inspect if contracts are still fulfilled or not
- => detached testing possible, heavily automated via CI/CD
- (still need for end-to-end tests)
CDC add to preventing failures to make their way into production

Interview D:

Advises to use CDC testing (over traditional integration tests)
- if you want to be truly independent
- need to think about mocks and stubs

Consumer-Driven Contract Testing

Context​

Problem​

Solution​

Example​

API Consumer Side​

API Provider Side​

Maturity​

Sources of Evidence​