Microservices as a self sufficient concept
Over the past few years microservices became a very hot topic in development communities. More and more people talk about the approach and attempt to build large software systems according to various patterns published by enthusiasts. The buzz words (and buzz concepts) are a big driving force in software development, which in a lot of cases results in failures, as people simply don’t research and learn the subject deep enough to be able to make informative decisions. I think one of the concepts suffering from such high level approach is distributed systems development, which some are trying by going with microservices model.
I am greatly interested in software architecture for distributed enterprise-level systems and I believe that microservices concept has not only been greatly misunderstood by many, but also is flawed in the way it is presented – as a self sufficient “architecture”. However, microservices is merely an implementation approach and without proper fundamentals will not magically lead you to success. What are those fundamentals you may ask? Well, read on – this is not going to be very long, even though each of the fundamentals is a very large subject.
Fundamentals
Service Oriented Architecture
You may have heard about SOA, but unless you’ve dug deep into this architectural style, your knowledge is most likely constrained by Web Services that were promoted by few major IT giants (Microsoft, IBM, Sun, etc) in early 2000. Web Services is really a bastardized view on SOA and doesn’t take into account some major concepts, so it’d be safe to say that any preconceptions you may have based on Web Services needs revisiting.
What are the most important concepts put forth by SOA?
Service ownership of data and behavior
A Service is a technical authority over particular business capability. This is a very important definition and though it is short it does take a while to fully grok the meaning. Any architecture is about understanding business processes and modeling those. Being an authority means that a service owns any data and any behavior associated with this business capability – no one else may have any knowledge of data or logic within a service. All that other services know is a contract announced by a service.
Autonomy
A Service is a self sufficient logical unit that is capable of fulfilling its capability without a need to ask any other service to do anything or to provide any data. This is what enables low coupling and results in a resilient system – a failure of a single service doesn’t lead to a failure of the system at large, but simply degrades functional availability for some business processes.
There is more, way more to SOA, but I will stop at these major points and will follow up on SOA in a separate post.
Event Driven Architecture
This architectural style promotes a design where system components notify the rest of the system of change in their state via events. An event is a notification, is a statement of something that has happened. Event driven design can scale from simple components compiled and running within a single executable to large distributed systems spanning tens or hundreds of executables. Obviously the large the system the more obvious the benefits, but the concept stays the same regardless of the size.
Domain Driven Design
Another huge subject that was postulated by Eric Evans. This is really a philosophy of how to approach architecture, design and implementation of software systems modeling complex (and in most cases large) business domains. There is a lot that can be said about DDD and a lot has been written on the subject, however at this time we are mostly interested in few tactical patterns (with the caveat that you really understand the importance of strategic patterns and understand when, why and how to apply the tactical ones).
Bounded Context
Bounded Context is a conceptual boundary where a domain model is applicable. Inside this boundary all terms and concepts have singular, specific meaning.
Aggregate
Aggregate in DDD is really all about being a boundary for transactional consistency – any change in state happens within the aggregate and owned and managed by it.
DDD concepts, actually, can be projected onto SOA concepts with a high degree of precision, which provides a much greater understanding and appreciation of the natural fundamentals behind large, complex systems. We shall look at how DDD and SOA concepts can be correlated to enrich our understanding of the subject domain and aid in implementation.
Command and Query Responsibility Segregation
CQRS is an architectural style invented and popularized by Greg Young. It postulates that there are two major functions of any software – changing state and reading, reporting, querying for state. It also postulates that using a single model for these two very different functions is not the best way to approach the solution.
Is Microservices enough?
As you can see, there is a fair bit of fundamentals that really take part in distributed systems architecture. And it is these fundamentals that really contribute to a successful architecture of large, complex, distributed software systems. Without understanding these fundamentals, the concepts they postulate, the constrains they define any attempt to design and implement a software system using microservices approach will, most likely, result in a failure.
Like I said before, microservices is really an implementation technique and needs to be used as such. Architecture and design should be fully based on SOA, EDA, DDD and, as you will see with time, CQRS architectural styles.
Bringing together SOA with EDA enables us to have a system where services do not directly talk to each other, but rather employ notification based style of communication. As you will see, there is really very little need for any other style of communication because each service is fully responsible for its operation. In those cases where we do want to ask a service to do something we would rely on an SOA concept of Command, which is an instruction.
Overlaying SOA and DDD we gain a greater in-depth knowledge of service boundaries (DDD bounded context), data and logic ownership and encapsulation. DDD Aggregate helps us in figuring out the transactional consistency boundary and naturally leads us towards the size and the scope of an implementation – this is where a microservice (or set of microservices) starts to appear in our picture.
Conclusion
I explicitly wanted to be brief and give a very high level opinion on why I think microservices without the correct foundation and fundamentals will not lead to easy wins or successes. In fact I believe that without understanding the fundamentals and architecting with them in mind any relatively involved and complex microservice based implementation will ultimately lead to a failure.
As a matter of fact, just recently an article was brought to my attention which very prominently highlighted that without good knowledge of these architectural fundamentals it is all too easy to find yourself in trouble while applying this “new, shiny” thing called microservices. The author of the article I mention talks about various complexities associated with microservices approach – all of which are avoidable completely or very simplified if fundamental architectural styles are understood and employed by architects. I am adding a post that acts as a response to the most prominent claims the author makes and I hope that another view on the subject and on those aspects will provide a fuller picture to readers.
A Kumar
I believe your points are very pertinent, and definitely do counter the ‘microservices death’ article you have referred to. I am indeed trying to understand the view points expressed in yours and the other article.
I must confess that I wish you had posted a few examples – even at the cost of lengthening the article. A lengthy article when full of useful knowledge is simply a delight that goes on longer!
Some specific questions:
1) You say a service will generally not need to communicate with another service. I find this difficult to believe. For example, “generate bill” or “accept payment” service will need to look up a user’s various details, including subscriptions etc., which will almost always be another service call. AND this will possibly be across aggregates. For example, calculation of loyalty points may involve getting details across various business departments – and I believe these departments have a fairly good chance of mapping to what you call aggregates
2) A direct call between services can often be replaced by a notification event. I do believe that the event based design promotes higher scalability – but at the cost of increased complexity, and possible splitting of a transaction into two or more transactions. For example, when a user purchases an item, his or her profile may or may not change depending on the $ amount of purchase. So item purchase and loyalty points change may be two different services, and they could communicate via notifications.
Design options A and B:
A) Create two services – item purchase, and loyalty adjustment. Both are separate transactions. In an ideal state, item-purchase would be immediately followed by loyalty adjustment. But to account for failure cases, a notification/pending-transactions table can be used to ensure that all the linked transactions are complete (via a batch job at the end of the night) . Now we have quite a few added complications caused by the event based design.
B) Create 3 services – item purchase, loyalty adjustment, and item-purchase plus simultaneous loyalty adjustment. Now all the 3 services can avoid distributed transactions … depending upon the schema design. But now we have duplicated functionality in the 3rd service.
What is your view on the above?
Alexander Yermakov
Hi Kumar,
This is a vast subject and there are hundreds of books on this, so it’s not really viable to have a “somewhat longer article that explains it all clearly”, unfortunately.
1. I speak of SOA Services / DDD Subdomains when I say there is hardly any need for point-to-point communication. However, there is always communication, but in most cases it follows notification style – eg. messaging in the form of “notification of what has happened” (aka Domain Event, aka SOA event). There are command messages as well – these carry an instruction to initiate an action.
2. There is not enough context around “generate bill” or “accept payment” for me to comment on what form these can / should take. These may be commands or synchronous service invocations from UI initiated by a user.
3. Departments won’t map to Aggregates. They *may* map to SOA Services / DDD Subdomains / DDD Bounded Contexts, but never to DDD Aggregates. DDD Aggregate is a transactional consistency boundary construct, and as such has very different purpose.
4. Changes to a profile based on a purchase or loyalty points logic live is a completely different subdomain from the one that owns purchases. Eg. upon a successful purchase, a notification is emitted by the Sales SOA Service. This notification is handled by Profile Service and by Loyalty Program Service and these then take action based on their logic and received information about a purchase.
5. Item Purchase and Loyalty Adjustments are *not* services. These are business processes.
6. Nothing in real world require a successful purchasing transaction to be cancelled and rolled back if your loyalty program is unavailable at that moment. Think of your regular grocery store visit – you either get your points or are asked to stop by Customer Service desk to get those points credited to your account later. You are *not* denied to carry out your milk / bread.
Another example. Amazon. You place your order, they take it. If they are out of stock all of a sudden (due to eventual consistency) – you get notified. Or they cancel the purchase after.
7. You should *never* have distributed transactions for couple of reasons:
– you will never make it fully work
– DDD aggregate is your transactional consistency boundary – so the transaction completes within it and you are always sure about the outcome of that.