Sponsor: Do you build complex software systems? See how NServiceBus makes it easier to design, build, and manage software systems that use message queues to achieve loose coupling. Get started for free.
If you’re working in a distributed application, you’re bound to run into a design issue where you want data consistency between services. But you don’t have a distributed transaction, so what’s the solution? In this video, I will take an example use case and explain the design challenge and solutions for handling communication and consistency between services.
YouTube
Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.
Workflow
This example use case was asked in my private Discord server by a member of my blog/channel. The domain is a subscription service where you buy a subscription and receive orders daily. The Subscription has a Balance for the amount your credit card charged. Your credit card is charged at an interval to keep a positive balance for your subscription.
One crucial aspect is that a subscription can lapse. Meaning that maybe the customer’s credit card has expired, and the customer did not update it in time before there was a $0 balance. Because of this, they allow a grace period of 2 days which you will still receive your Order, and you’ll go into a negative balance. Once you update your credit card and your credit card is successfully charged, the balance is updated, removing you from a negative balance and also updating the orders received during the grace period as being paid.
Here’s how the current system works when a subscription lapses.
There are two boundaries. One boundary is for handling credit card payments and managing subscriptions. The other boundary is for managing Orders.
The customer updates their credit card, and the Payment services hit the payment gateway to charge their credit card.
Once the credit card is charged successfully, they update the subscription to set the new balance to the amount charged.
Then they use an event-driven architecture, create a PaymentCompleted Event, and publish it to a message broker.
The Order service consumes that event. It looks at its database to determine which orders have not been paid yet.
Then the order service makes a synchronous blocking RPC call (HTTP or gRPC) back to the Payment service to decrease the balance for the orders marked as paid, which were created during the grace period.
Once that RPC call is completed, the order service can mark the orders as fully paid. Now both Order service and Payment service are consistent. All the orders are marked as paid, and the payment service’s balance is correct and consistent.
But what happens if there is an error at that last step, updating the Orders status as paid?
Now we’re left in an inconsistent state. We’ve decreased the balance in the Payment service but failed to set the Orders as paid.
It sure looks like we need a distributed transaction! Not so fast.
Boundaries
Another solution is not needing a distributed transaction. We have these consistency issues because we are keeping track of the order status separately from the balance.
The Orders Service contains the status of the order. The subscription service has the subscription balance and all the credit card transactions.
One solution is to move the status of an order to the subscription service. This means having the same concept of order in both boundaries but for different purposes. They only share the OrderId and the amount. This is a simplified example but there might be many more pieces of data that are unique to each boundary. For example, the Orders boundary also has the CustomerId, which the Subscription boundary doesn’t care about. It cares about the status and the amount of all the orders.
These changes now mean we don’t have to communicate or have any workflow between services when a credit card is updated, and we need to mark orders as paid that were created during the grace period.
When the credit card is updated, we hit the payment gateway to charge the customer’s credit card.
Then we update the balance as we did before; however, now we can also, within the same transaction, update the orders that have not been marked as paid because we have the order status within the Payment service.
Entity Services
Why did the Order service need to own the order status, determining if it was paid? This is because we often get caught up in entity services. Services that own everything to do with an entity. However, the entity usually has different purposes for different boundaries. You do not need to have a single Entity live in only one Service. The concept of an entity can exist in many different boundaries, and each owns a portion of the data and behaviors around that data.
Do you need a distributed transaction? Maybe not. Look at data ownership around the consistency you’re looking for.
Join!
Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.