Category: Software Design

If you’re working in a distributed application, you’re bound to run into a design issue where you want data consistency between services. But you don’t have a distributed transaction, so what’s the solution? In this video, I will take an example use case and explain the design challenge and solutions for handling communication and consistency between services.

YouTube

Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.

https://www.youtube.com/watch?v=Trl4–FO7Yo

Workflow

This example use case was asked in my private Discord server by a member of my blog/channel. The domain is a subscription service where you buy a subscription and receive orders daily. The Subscription has a Balance for the amount your credit card charged. Your credit card is charged at an interval to keep a positive balance for your subscription.

One crucial aspect is that a subscription can lapse. Meaning that maybe the customer’s credit card has expired, and the customer did not update it in time before there was a $0 balance. Because of this, they allow a grace period of 2 days which you will still receive your Order, and you’ll go into a negative balance. Once you update your credit card and your credit card is successfully charged, the balance is updated, removing you from a negative balance and also updating the orders received during the grace period as being paid.

Here’s how the current system works when a subscription lapses.

There are two boundaries. One boundary is for handling credit card payments and managing subscriptions. The other boundary is for managing Orders.

The customer updates their credit card, and the Payment services hit the payment gateway to charge their credit card.

Once the credit card is charged successfully, they update the subscription to set the new balance to the amount charged.

Then they use an event-driven architecture, create a PaymentCompleted Event, and publish it to a message broker.

The Order service consumes that event. It looks at its database to determine which orders have not been paid yet.

Then the order service makes a synchronous blocking RPC call (HTTP or gRPC) back to the Payment service to decrease the balance for the orders marked as paid, which were created during the grace period.

Once that RPC call is completed, the order service can mark the orders as fully paid. Now both Order service and Payment service are consistent. All the orders are marked as paid, and the payment service’s balance is correct and consistent.

But what happens if there is an error at that last step, updating the Orders status as paid?

Now we’re left in an inconsistent state. We’ve decreased the balance in the Payment service but failed to set the Orders as paid.

It sure looks like we need a distributed transaction! Not so fast.

Boundaries

Another solution is not needing a distributed transaction. We have these consistency issues because we are keeping track of the order status separately from the balance.

The Orders Service contains the status of the order. The subscription service has the subscription balance and all the credit card transactions.

One solution is to move the status of an order to the subscription service. This means having the same concept of order in both boundaries but for different purposes. They only share the OrderId and the amount. This is a simplified example but there might be many more pieces of data that are unique to each boundary. For example, the Orders boundary also has the CustomerId, which the Subscription boundary doesn’t care about. It cares about the status and the amount of all the orders.

These changes now mean we don’t have to communicate or have any workflow between services when a credit card is updated, and we need to mark orders as paid that were created during the grace period.

When the credit card is updated, we hit the payment gateway to charge the customer’s credit card.

Then we update the balance as we did before; however, now we can also, within the same transaction, update the orders that have not been marked as paid because we have the order status within the Payment service.

Entity Services

Why did the Order service need to own the order status, determining if it was paid? This is because we often get caught up in entity services. Services that own everything to do with an entity. However, the entity usually has different purposes for different boundaries. You do not need to have a single Entity live in only one Service. The concept of an entity can exist in many different boundaries, and each owns a portion of the data and behaviors around that data.

Do you need a distributed transaction? Maybe not. Look at data ownership around the consistency you’re looking for.

Join!

Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.

Related Links

• AVOID Entity Services by Focusing on Capabilities
• SOLID? Nope, just Coupling and Cohesion
• REST APIs for Microservices? Beware!

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

Leave this field empty if you're human:

Should you care about design patterns? There are books devoted to them; heck, even I post videos about specific design patterns. But do they matter? If you’re new to design, it can be overwhelming and cause a lot of unneeded complexity. I will cover how I think of design patterns or how I don’t think of design patterns.

YouTube

Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.

https://www.youtube.com/watch?v=V-2Ad5zg3D8

Bell Curve

In my own personal experience, and witnessing the same occur to other developers in their careers, is a mid-career explosion of complexity. Design Patterns play a big role in this explosion of complexity.

Early on in a developer’s career, they are often righting pretty simple code. While that code may be highly coupled and flawed, they generally write “simple code.”

Now we can argue what “simple code” means, but speaking for myself, it was straightforward without any magic. You aren’t writing “smart” or “clever” code. What you see is what you get. There was little indirection.

As you gain more experience and read or watch various tutorials/courses/books, you start seeing every problem as a means to solve by patterns.

There is a similar case to be made for doing the same thing with the latest technology, library, framework, or platform. You learn something new and immediately want to apply it. Unfortunately, this often leads to using it aimlessly. Meaning you have a hammer, so everything starts looking like a nail.

Now you might be thinking, “really, people are just applying patterns for no good reason?”. Yes, this is more common than you think. It can also be because you’re applying patterns because you think you have the problem it solves. More on that later.

Hopefully, you feel enough pain in this phase of your career where you can realize you aren’t solving problems but rather creating unneeded complexity. On the bright side, you’ll better understand various patterns and the problems they solve, as you’ve used them for the right and wrong reasons.

On the other side of the complexity, the nightmare phase is back to the simplicity that resembles the naive code you’ve written at the beginning of your career.

That’s not to say you’re writing beginner code, but it’s focused, trivial, with no magic, and less useless indirection. It’s direct and to the point. As someone commented on the YouTube video:

It took me four years to paint like Raphael, but a lifetime to paint like a child.

Pablo Picasso

Communication

A key to patterns is communication. Named patterns are a way to communicate between developers about solutions and implementations for various problems. For example, one developer is explaining a problem they are having with another developer. The other developer says that the problem could be solved by [Insert Named Pattern]. If both developers understand the named pattern, they don’t have to get lost in the deep implementation details of that pattern. They already understand it. It’s a communication tool.

According to many comments on various videos I’ve done on YouTube, it’s common for people to apply a pattern without even realizing it’s a named pattern. I have done this countless times over my career. You’re faced with a problem and come up with a solution that turns out to be a named pattern! Once you realize this, great! You understand both the problem the pattern truly solves and how to implement it.

Knowing the names of patterns and the problems they solve is great for communication.

Avoiding the Problem

Earlier I mentioned that you could apply patterns for problems you think you have. However, it’s often helpful to examine why you have the problem. Meaning one solution is to avoid the problem in the first place.

To illustrate this, I’m going to use the example of the Repository Pattern. Now you could take this example more abstractly and apply it to other patterns that add indirection.

There are different definitions of the repository pattern, but for this example, I’ll say it’s used to encapsulate data access logic.

Mixing data access logic with other concerns sounds like a terrible idea. However, there is an underlying issue that’s not talked about. Coupling.

With the repository, you’re coupling to it rather than likely using a native database provider directly. The purpose of abstractions is to simplify the interface for your purpose. The repository pattern can do this for us. Great. However, you still have the same degree of coupling from your application code using the repository.

If you have hundreds (or thousands) of usages of the repository in your application code, you have a high degree of coupling to your repository. If you make any breaking changes to your repository, you’re faced with changing all calling code that breaks.

Another solution is to limit coupling.

In many situations, it’s not that you need all the data or implementation of the repository. Often you only need a subset. The repository might not be ideal in every situation. I talked about this in my post Should you use the Repository Pattern? With CQRS, Yes and No!

Using your repository abstraction might be helpful in one situation and not ideal in another. You may decide a subset of features is grouped together, use the same related data, and use a repository. Another subgroup of features might choose to access data differently because of its use case.

The problem isn’t that you need to encapsulate data access; the problem is you have a high degree of coupling of calling code that needs data access. One solution is to encapsulate data access with a repository. Another is to limit coupling, so the need for encapsulating data access is less of a concern or decided per situation.

Design Patterns

Should you care about design patterns? Yes, absolutely. Understanding the names of design patterns and the problems they solve is helpful. It’s a great way to communicate with other developers when discussing problems and solutions. However, don’t apply a pattern unless you truly have the problem it solves. Do you have the problem it solves? Or should you look at ways to eliminate the problem.

Join!

Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.

Related Links

• SOLID? Nope, just Coupling and Cohesion
• STOP doing dogmatic Domain Driven Design

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

Leave this field empty if you're human:

Guard clauses, on the surface, sound like a great idea. They can reduce conditional complexity by exiting a method or function early. However, I find guard clauses used in the real world to be of little value. Often polluting application-level code for trivial preconditions. I will refactor some code to push those preconditions forcing the edge of your application so your domain focuses on real business concerns.

YouTube

Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.

https://www.youtube.com/watch?v=9cr7grNWn6c

Null Checks

The most common case of guard clauses is doing null checks. To illustrate this, I looked at the eShopOnWeb sample application and will use it as an example.

In the example above, two guard clauses do null checks on the anonymousId and userName. While this is incredibly common, I’d rather not have to deal with these preconditions mainly because this method is in the BasketService and a part of the core application code.

Often these types of preconditions are very inconsistent. Since the userName is likely passed around through various layers, does each method that accepts the username have this guard clause? Likely not.

As an example, the above code creates a new instance of the Basket passing the userName to the constrictor. Here’s the constructor.

Sure the TransferBasketAsync method was doing the null check, but does this Basket class get created anywhere else? Is it doing a null check? As you can imagine, if you did this everywhere, you’d have a ton of repetitive trivial code for these null check preconditions.

Forcing Valid Values

I don’t want to litter my core application code with trivial guard clauses, such as null checks, as my example. Instead, I want my core code always to accept valid values so that it doesn’t need to concern itself with doing these guard clauses.

In doing so, you’re pushing the responsibility to the outer edge of your application to produce valid values. If you think about a web application, there is some translation from an HTTP request into your application code. You want to force that translation at the edge, which is your web endpoint, into valid domain values.

One way to accomplish this, as in my example with the userName is to define a type (a record struct) that, during creation, forces the value not to be null.

Then we can use this type wherever we accept the userName as a parameter. Instead of accepting a string for the anonymousId and username in the TransferBasketAsync method, we can move this to the Username type.

To call this TransferBasketAsync in the BasketService, you must construct a Username type. In this example, this is done on a Razor page.

In the above example, the userName comes from the HTTP request, which will be a string. We then construct a Username type passing in that string value. We’re pushing the validation to be at the edge of the application.

Our core application defines the Username type, but its usage/creation is at the edge. Nothing can get into the core application without being in a valid state.

Tests

Because of guard clauses, such as null checks, there tend to be tests associated with them. This was the case with this eShopOnWeb sample, where there were tests to confirm those null checks were done. But since we’ve moved to a type that forces it to be valid, these tests still passed because it was throwing. However, these tests are useless now, and we can remove them. We can now remove any tests that were related to doing a null check against the string username.

Instead, we have a few tests to confirm the behavior of our new Username type.

Primitive Obsession

As some others commented on the YouTube video, you might be thinking that I’ve introduced a value object to combat primitive obsession. While true, that’s not the seeing the root cause. The root cause isn’t primitive obsession. The issue is allowing your domain to accept invalid arguments that you need to guard against. This can apply to primitives that are null, as my example illustrates, but it can also be explicit invariants for generic examples, Money, or Date/times with Timezones. However, this can include very domain-specific values.

Guard Clauses

I’m not suggesting guard clauses are not helpful. They are. Exiting early within a method when preconditions aren’t met simplifies logic. However, littering trivial guard clauses all over your codebase is not helpful. Force the outer edge of your application to construct valid values that are passed into your core domain code.

Join!

Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.

You also might like

• Just store UTC? Not so fast! Handling Time zones is complicated.
• STOP Over-Engineering Software!

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

Leave this field empty if you're human: