When NOT to write an Abstraction Layer

Common advice is to abstract dependencies. Often this is because people have been burned by depending directly upon 3rd party dependencies that they are highly coupled to. If they need to change the dependency, which they are highly coupled with, it can be pretty painful. This is why people say to create an abstraction layer around any dependency to isolate it so that your application code is coupled to yourself rather than a 3rd party. While this is generally true, it depends, and you don’t want to create an abstraction that is more work in the short and long run by creating the wrong abstraction. Here’s a concrete example of when you shouldn’t abstract.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything that is in this post.

Abstraction Layer

So why is the general advice to abstract 3rd party dependencies? Well if you’re going to be highly coupled to the dependency, and it has breaking changes (via new releases/versions), then you’re going to have to deal with all the breaking changes everywhere you’re using that dependency in your application code.

To alleviate that issue, one solution is to create your own abstraction layer that your application code depends on instead. This way if there is any change to the underlying dependency, you are changing it within the implementation of your abstraction.

Generally, this abstraction is a wrapper (facade) that hides the complexity of the 3rd party and often times simplifies its usage for your specific use case(s).

A common example to think of is the Repository Pattern. You’re creating a repository to abstract the native interactions with the underlying database. These interactions could be from a 3rd Party ORM or even using the native SDK for the database you’re using.

Abstraction Layer

In this case, your application code is not directly coupled to the ORM (dependency) but rather your application code is coupled to your repository. If anything changes with the underlying ORM (or SDK) then you must make changes to the repository rather than in any application code.

I’ve talked about the repository pattern before, you might want to check out my post Should you use the Repository Pattern? With CQRS, Yes and No!

Messaging

While creating abstractions around 3rd parties is generally a good idea in a lot of situations, it’s not a very good idea in others. A concrete example of this is with a messaging library.

If you’re using a specific message broker directly with their SDK, then you likely will create for the same reason create an abstraction around it.

Abstraction Layer

However, once you get into the deep end with messaging and are using an Event (Message) Driven Architecture, you’ll start implementing common patterns and concepts.

Common patterns and concepts such as the Outbox Pattern, Fault Tolerance, Scheduled (Delayed) Delivery, Claim Check, Encryption, Stateful(less) Process Managers, and more.

However, there are messaging libraries that already do this. They are themselves abstractions over various queuing technology. Meaning you can use a messaging library that would support RabbitMQ, Azure Service Bus, Amazon SQS, and others.

Messaging libraries also implement and expose ways to use those patterns and concepts that are very common when in an Event (Message) Driven Architecture.

So the question becomes, should you abstract a messaging library?

Abstraction Layer

Opinionated

The problem is these types of libraries are very opinionated on how they implement common patterns and concepts. This means that if you create your own abstraction over a messaging library, you’ll inherently create something that is very specific to the actual messaging library.

If you need to expose something like the outbox pattern in your own abstraction, then depending on how the underlying messaging library accomplishes that, you’ll have to expose that in a very similar way.

Meaning you’ll likely leak some details that are not implemented the same way within another messaging library.

Your abstraction will have the same similar API surface and opinions about implementation as the underlying messaging library. This means your abstraction will be very similar to the point of adding no additional value other than hiding the dependency.

Abstraction Layer

Now if you want to change the messaging library dependency with another, you might think that having your abstraction would be useful. But since it’s very opinionated, they don’t really line up in terms of how functionality is exposed. I purposely made the two Messaging Library boxes in the diagrams different shapes for this reason.

You won’t be able to fit your abstraction over a different dependency because both dependencies are different in how they accomplish the same patterns and concepts.

I’m not against abstractions or creating your own abstraction layers over dependencies. I do think in general rule is a decent approach in a lot of situations, just not all situations. Creating your own abstraction so you depend on your own types, often facades that isolate and simplify the underlying dependency for your use cases.

Complexity

In the concrete example of a messaging library, it doesn’t make sense to create an abstraction layer. A messaging library is itself an abstraction. It will be core to a system that is built around Event Driven Architecture and messaging. There will be little value gained by creating your own abstraction but rather more complexity added by adding indirection.

Source Code

Developer-level members of my YouTube channel or Patreon get access to the full source for any working demo application that I post on my blog or YouTube. Check out the YouTube Membership or Patreon for more info.

Related Links

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

Long live the Monolith! Monolithic Architecture != Big Ball of Mud

If you’re developing a Monolith or using a Monolithic Architecture doesn’t mean it needs to be a big ball of mud. Most people equate a Monolith with a Big Ball of Mud because it’s highly coupled and difficult to change. However, you can combat it by defining strict boundaries and logically decoupling those boundaries and the data that each boundary owns. To go even further you can loosely couple by leveraging asynchronous messaging between boundaries. Does this sound familiar? Like Microservices where each service has its defined capabilities and database?

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything that is in this post.

Coupling

When people think of a Monolith or Monolithic Architecture they call it a tangled, spaghetti code mess that’s really difficult to make changes. What is really the issue is that the majority of the system is tightly coupled.

Below is a dependency graph that illustrates how each component/class/module is connected to others.

Systems that have many modules that have a high degree of afferent and efferent coupling are bound to be brittle. Changes to one module can have cascading effects on all dependant modules. Coupling is a necessary evil but can be managed.

Coupling exists equally with a microservices architecture. If you’ve developed a microservices architecture that relies heavily upon RPC to communicate between services, the coupling is no different than a monolith that communicates in process. Adding the synchronous calls over the network between services doesn’t magically make coupling go away. You’ve simply developed a distributed monolith, not microservices. If anything, communicating over the network via RPC makes things worse. Why? Check out my post REST APIs for Microservices? Beware!

Boundaries

Defining service boundaries is one of the most important aspects of designing a system, yet getting them “right” is incredibly difficult. There are a lot of tradeoffs that determine where those boundaries should lie.

Services should own a set of business capabilities and data. You may have services that other services boundaries need for query purposes, but a single service should own that data because of the capabilities it provides.

If you think about an existing monolithic architecture or a distributed monolith (bad microservices), let’s represent it by this large turd pile (poop emoji).

What you want to develop are smaller turd piles. Decomposing a large coupled system into smaller units. Each is a service boundary.

Another way to visualize this is to think of a piece of cake as a large high coupled system. The cake may be using a layered architecture but still has a high degree of coupling. To decompose you want to cut out a piece of the cake.

For more info on defining services boundaries, check out my post Context is King: Finding Service Boundaries

Loosely Coupled Monolith

The structure of a monolith that has well-defined service boundaries means that each boundary must first own its own data. If you’re developing a monolith this means that you may still have a single database instance, but the schema (tables, collections, etc) is only accessed by that boundary. No other boundaries are able to access them directly from data storage.

Obviously, boundaries will need to communicate so they can interact with each other. If you want to minimize coupling between boundaries, then a solution is to use asynchronous messaging. This means introducing a message broker that your monolith is sending messages to and also receiving.

Messages are contracts. They are simple DTOs that represent Commands and Events. This is why in the above diagram you see that the implementation for any service boundary is only coupled to the contracts of other service boundaries.

As an example of creating an HTTP API, it would host all the different service boundaries. each defines all its routes and dependencies within it. It can send commands and publish events to the message broker. Simply for scaling purposes, I’ve illustrated another top process as the Message Processor. It’s using the exact same code as the HTTP API. It will be connected to the message broker to consume messages and then dispatch them in-process to the appropriate boundary for the messages to be handled.

As with any monolith, if you need to communicate synchronously between boundaries, you can do so behind interfaces and functions which live within the contracts projects.

Deployment

The difference between a loosely coupled monolith and microservices at this point becomes a deployment concern. Each has well-defined boundaries and communicates primarily via asynchronous messaging. If you needed to separate a boundary within your monolith because you want to scale it differently or want it to be independently deployable, then you can go down that road, but with added complexity. Check out my post on scaling a monolith horizontally before you go down that road, however.

Monolithic Architecture

A monolith does not need to be a big ball of mud. It does not need to be a highly coupled pile of spaghetti that is difficult to change. Define service boundaries and limit direct synchronous communication as much as possible while leveraging asynchronous messaging.

Source Code

Developer-level members of my YouTube channel or Patreon get access to the full source for any working demo application that I post on my blog or YouTube. Check out the YouTube Membership or Patreon for more info.

Related Links

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

What’s the Cost of Indirection & Abstractions?

Indirection is fundamental to software design. Creating abstractions is one common way of creating indirection. The benefits are reuse, isolating complexity, encapsulation of dependencies, and more. But what’s the cost of indirection & abstractions? Cognitive load to fully understand all of the layers of a request and limiting functionality.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything that is in this post.


Indirection

To illustrate indirection, first, let’s get down to the basics of having calling code (Caller) that is invoking another piece of code (Target).

Adding indirection simply means adding something in between the caller and the target.

As developers, we’re adding indirection all the time without really thinking of it. It’s not inherently bad and comes with a lot of benefits. Adding indirection is a useful way of isolating complexity, allowing re-use, and abstracting dependencies as we can have many different callers use the same abstraction.

A typical example of indirection is adding an abstraction of data access. This can often be thought to if you’re creating some type of data abstraction layer, a repository, etc.

If you’ve ever used the decorator pattern or created a request pipeline (check out my post on Separating Concerns with Pipes & Filters), you’re creating indirection.

Another way to think about indirection which isn’t directly related to application code is infrastructure. Indirection can come from a message queue, topics, load balancer, etc.

Code Example

I’m going to be showing snippets from the eShopOnWeb reference application. Below is a from the OrderController that has a route to show the signed-in users Orders.

The first use of indirection in our code is using MediatR. Instead of having application code in our ASP.NET Controller, we’re using MediatR to invoke our Query Handler which will have that logic.

The second use of indirection is we’re injecting an IOrderRepository and using that to get out the list of Orders. Meaning our indirection is coming from separating data access. When then take that list and transform it into a list of OrderViewModels that is returned from our Handler.

The OrderRepository is actually adding another layer of indirection because it is using Entity Framework Core within it to get data from the database.

So the call stack from the Controller to our database looks like this:

What's the Cost of Indirection & Abstractions?

If we removed most of the indirection it would look like this:

I’ve left Entity Framework as a layer because ultimately you’d be using some type of data access client to get to the database, regardless if that’s Entity Framework or simply ADO.NET directly.

Now I’m not implying you should remove indirection! There are clear benefits. To start with, the usage of MediatR can be benefiting from not coupling your application code with ASP.NET Core. Let your application code focus on application logic and let ASP.NET Core handle HTTP.

In the case of the repository, its purpose is to abstract the dependency on Entity Framework Core. Instead of having application logic directly couple to a 3rd party dependency, creating an abstraction of the repository allows you to couple to a type that you own (although I’ll argue later I don’t have to).

Cost of Indirection & Abstractions

The first cost of indirection is cognitive load.  If you need to understand the full life of a request and everything that happens, depending on how many layers the request is passing through can be challenging.

In the example above, it’s pretty simple however you can imagine the more indirection that exists, the more difficult it will be to understand the full scope.

On the flip side, there can be the benefit of not having to worry about certain layers.  Meaning you simply don’t have to concern yourself with them.  Until you do.

My point is keeping indirection to a level where you have the ability to fully understand the entire request and how it pertains to the application code you’re writing.

The second cost is performance because of limited functionality.  Not necessarily from a memory allocation or CPU perspective, although that’s possible, more because when you’re creating indirection through abstractions, you’re often times making your abstraction generic or a limited surface of what we’re abstracting.  This occurs often when you’re abstracting a 3rd party dependency.

To illustrate this, look back at the Handler that was using the repository.  Does it really need to use a repository? what effect does using the repository have?

To get all the Orders out of the Repository, it was taking a Specification. This is the specification it was using. Its purpose is to add the Where() so it’s only fetching the orders for a specific user and to eagerly load the OrderItems and then the ItemOrdered which is the actual product.

Here’s the IRepository, which if you’ve used a repository before, probably looks pretty familiar.

This is all very generic and limits the ability to really leverage the underlying data access, which is Entity Framework. Because we’re abstracting entity framework behind this Repository, we’re now stripped out a bunch of functionality that we can’t expose.

Because of this, the listing page is getting back way more data than it actually needs.

The repository is returning line items and for each line item the associated product. None of this is used within this view.

Personally, I’d rather not use a repository in this situation. Why? Check out my video on Should you use the Repository Pattern? With CQRS, Yes and No!

Indirection is something we’re constantly creating but it has a cost. Be aware of when you’re adding indirection and if it actually adding value. If you’re abstracting a dependency so you can make it more testable, then great. If the dependency is testable and you simply don’t want to directly couple to it, then that might make sense, or it might not!

Source Code

Developer-level members of my CodeOpinion YouTube channel get access to the full source for any working demo application that I post on my blog or YouTube. Check out the membership for more info.

Related Links

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design