Skip to main content

Data Replication Proxy

Context​

The system is split into multiple microservices with corresponding APIs. There is at least one 3rd party application (e.g. an external API, a CMS, ...) or a legacy system during a migration that shall be integrated into the system.

Problem​

  • In a migration scenario, a big bang integration or parallel adoption is too much risk
  • Integrating 3rd party applications or the legacy system leads to an inconsistent integration landscape (e.g. different communication protocol)
  • The integrated 3rd party application or legacy system does not live up to the availability guarantees required for a microservice-based system

Solution​

Replicate data of the 3rd party or legacy system in the proxy microservice to build up a cache. By keeping a replica of the data in the microservice, the load on the wrapped system is reduced and availability guarantees can be given (because it is just another microservice).

Context-dependent questions to be answered are the following:

  • Which cache should be used? (in-menory, on-disk, database, ...)
  • How big should the cache be?
  • How do you update the change? (when forwarding to the wrapped system, after receiving an answer)
  • If cache updates are done before receiving a success message of the wrapped system, how to revoke the earlier changes?

An open challenge is keeping the wrapped system and the proxy microservice in sync to fulfill suiting data consistency guarantees.

This technique is a specialization of the proxy microservice technique.

Maturity​

More data is required (only 2 sources).

Sources of Evidence​

Note: As this technique is a specialization of the proxy microservice technique and an abstraction of the CQRS proxy, we also used their sources to formulate this technique.

L41:

  • Context: interaction microservices with a mainframe system (migration)
  • For now, impact of mainframe reduced by caching

Interview D:

  • Context: how to decouple from a legacy system
  • one solution: caching
  • questions to be asked
    • which kind of cache?
    • how big is the cache?
    • how up-to-date is the cache?
    • cache update before forwarding to legacy system and mark it as "might not be the truth", or even act like it will be true?
      • do I apologize when it doesn't become the truth? How? How to revoke it from the lest of the service landscape? => compensation actions
        • often too complex
  • Agrees that data replication to increase accessible makes sense
    • need for a cache layer between
    • doesn't matter if in-memory or on disk
    • => need for a decoupling by data replication