Why Data is the Hard Part of Working with Microservices

Microservices are popular among dev teams working on large, modular projects. They allow different teams to work at different paces on different parts of a software system. But working with microservices comes with large challenges that are too often not well appreciated. One of the greatest challenges, which developers too often underestimate, is working with data, argues Red Hat’s Christian Posta over at his blog.

The dilemma for microservice users is that you don’t want many different services sharing one database, but you do want ACID transactions and a single source of data. Christian argues, to solve this dilemma, you have to do domain driven design (DDD). You need to define domain models that are bounded by a context and the boundaries of each context define a microservice. So for example, for an online bookstore you might have a book in the context of books for sale in a library where the book model would include price, publisher, ISBN and so on. In a different setting you might have a book in the context of viewing its contents where you want the number of pages, author details and so on.

You might be saying to yourself: ‘wait, I never heard about any of this stuff in relation to microservices. Netflix doesn’t do this.’ Christian argues internet companies like Netflix don’t use microservices for the same reasons that traditional enterprises do. Netflix needs microservices to handle data volumes, traditional companies need them to handle complexity and that’s where the domain model helps.

The solution to the data dilemma for traditional enterprises is to define the transactional boundaries as tightly as possible. That is, only allow each microservice to make small, atomic transactions that enforce business constraints and worry about ensuring data consistency across the system later.

Christian recommends using events to communicate across boundaries. There will always be failures of communication between microservices but you only need to guarantee consistency at some point in time. So set up services that publish events about things happening in their domain to other services that can then adapt their models and persist any required changes to their databases.

Just keep in mind that microservices are just a form of distributed system with all its attendant problems: data, data integration, data boundaries, timing and more. 

Original Article

The Hardest Part About Microservices: Your Data

Seamus Holland