Why is Clean Architecture so Popular?

You’ve probably noticed many videos and blogs that somewhat explain what Clean Architecture is and show you how to use it. So its Clean Architecture is popular, but should it be? Should you be using it? Here’s why I think it’s popular, the problems it addresses, and some aspects that almost nobody ever mentions.

YouTube

Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.

Clean Architecture

Clean Architecture
https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html

As a quick primer, what is clean architecture? Well, it’s a way to manage coupling. Specifically, in this diagram, you can see how the outer parts of the circle reference the inner parts of the circle. The dependencies between layers are pointing in a single direction inward.

It’s about managing coupling.

As an example, with the Clean Architecture template for .NET/C#, the project structure and dependencies are as follows.

Clean Architecture Direction of Dependencies

The top (outer layer), called WebUI, is ASP.NET Core. It references an Infrastructure project that contains the entity framework DBContext and other concerns. The WebUI and Infrastructure reference the Application project, which contains the interfaces for implementations in the infrastructure and any application-level code, such as commands, queries, and handlers. Finally, the application project references the Domain project, which contains (or should) your domain models and business logic.

Sounds great. Separation of technical concerns. But why?

Coupling

degree of interdependence between software modules

ISO/IEC/IEEE 24765:2010
Systems and software engineering — Vocabulary

Big ball of Mud

There are two forms of coupling Clean Architecture addresses. Afferent and Efferent.

Efferent Coupling: Who do you depend on? From the perspective of the Domain project, who does it depend on? Nothing.

Afferent Coupling: Who depends on you? From the same perspective of the domain project, which projects depends on it? The Application Project.

This is about stability. Because the Domain project has no dependencies, nothing can force it to change. All our business logic is isolated and cannot be forced to change because of a change within the infrastructure project or any other project. The reverse is true for WebUI. Changes we make in the infrastructure or Application could force us to make changes in the WebUI.

Do you need Clean Architecture?

It would be best if you asked yourself a few questions. What is the size of the application? Do I have complex domain logic? Do I need to control coupling?

Clean architecture is about forcing a direction of dependencies. In .NET, projects were used in the template above to force the separation. However, you do not need separate projects. Coupling is the dependence between types. If you merged the template into one project, you still have the same degree of coupling.

Prescription

Do not use Clean Architecture as a prescription or template. Understand that you’re trying to manage coupling. It doesn’t need to be by projects. However, it can be to help with physical separation. It doesn’t need to be those exact layers. It’s not a prescription.

Large System

You should consider decomposing it into logical boundaries if you have a large system. What’s a large system? Something that takes a team of developers, possibly years to develop. I’ve covered this in many different blog posts and videos. Check out my post Microservices gets it WRONG defining Service Boundaries and Should you use Domain Driven Design? where I talk about logical boundaries. Logical boundaries are about grouping a cohesive set of capabilities within your system. It allows you to decompose a large system into smaller subsystems.

Logical Boundaries

Why does this matter? When you break up a large system into smaller parts, you’ll realize that not all parts provide the same value. While all the boundaries are important, some are more in a supporting role and often built around CRUD (Create-Read-Update-Delete). This is also very similar if you’re creating a smaller app that may take a couple of weeks or months to develop.

If you have no domain logic, do you need to all the same layers as another part of your system that is at the core of the solution space and contains complex business logic? No.

Clean Architecture within logical boundaries

This is why it’s not a prescription or template. Each boundary within a system has different concerns. If you don’t have any business rules, you have an underlying data model. Or perhaps you only have a dozen or so routes/endpoints that have data access. Do you need to add an abstraction to data access in that case? What if your database changes? Then change the 12 or so routes/endpoints!

Clean Architecture

Clean architecture is about coupling. There’s no prescription for the layers you define or how you define the coupling. You don’t need to define layers by projects. It’s about the direction of dependencies between types. Afferent and Efferent coupling are what define the stability of each layer. Do you need stability in a particular layer? Then maybe consider isolating it.

Join!

Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

McDonald’s Journey to Event-Driven Architecture

McDonald’s uses Event-Driven Architecture! Luckily for us, they’ve written a couple of blog posts providing some details of their journey into event-driven architecture. I’m going to go a bit deeper by providing my thoughts on how their system works and why they are doing it so that it can give you some ideas about your systems.

YouTube

Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.

McDonald’s Event-Driven Architecture

It’s always interesting to see companies post details of the architecture of various systems they have. It can be insightful to see what they are doing, why, and their challenges. McDonald’s posted behind-the-scenes and how-it-works blog posts detailing their journey to event-driven architecture. More specifically, it’s not that they are new to event-driven architecture but rather have a standardized way to implement it with distributed teams of developers with different skill levels.

McDonald's Event-Driven Architecture

There are many different components to their platform. Their infrastructure is within AWS, and they use MSK (Managed Streaming for Kafka) along with ECS, DynamocDB, and API Gateway.

Here’s how everything works together.

Schema Registry

One of their challenges was related to data quality. Likely because there was no set definition (schema) for data within events. If multiple producers produce the same event type, they might not be composing them exactly the same. I believe an event should have a single publisher, the owner of that schema, to avoid this issue. However, this could be applicable in a message-driven architecture that’s also using queues and commands.

Producers at startup use a custom SDK that retrieves all the event schemas from the registry. This allows the producer to validate the event being produced against the schema.

If validation passes, the producer can publish this event to the appropriate Kafka topic using the SDK at this point.

As you can expect, on the consumer side, the same thing occurs. Consumers at startup use a custom SDK that retrieves all the schemas from the registry, just like the producers do.

Then the consumers can process messages from the Kafka topics and understand how to deserialize them from the schema and version of the schema.

Everything within any Kafka topic should be valid based on all the schemas (versioned) within the registry. Data quality issues are solved!

Validation

Of course, not everything goes through the happy path. What if a producer tries to publish an event, but it fails to validate against the schema? The producer then publishes the message to a Dead Letter Queue. Kafka isn’t a queue, so this is a Dead Letter Topic.

Producer to DLQ

Once a message is in the “DLQ” there needs to be a way to view, modify and fix the event so it can be re-published to the correct topic.

For this, an Admin/Utility UI provides this functionality for them.

Reliable Publishing

The second failure that can occur is failing to publish to Kafka (MSK). Anyone getting involved in Event-Driven Architecture is bound to run into this. It would be best if you had consistency between making state changes to your business data and publishing your event. When events become critical to your system and possibly workflows, you need guarantees that you publish the relevant events when you make some state change to business data.

Mcdonald’s chose to use DynamoDB to persist any events that cannot be published to Kafka. This means their Publisher SDK will fallback to storing the event data within DynamoDB if it cannot publish to Kafka.

Using a fallback to some durable storage is a common approach. However, the Outbox Pattern is another common solution. I discussed this and other common issues in a post about the 5 Pitfalls of EDA.

Once the event data is in DynamoDB, they use Lambda to pull it from DynamoDB and then retry and publish it to Kafka. I’d assume they have different retry intervals/backoffs.

Lambda Retry

Gateway

Lastly, if you’re integrating with 3rd parties or even within a large organization, you’ll need to have them publish events. However, they won’t have direct access to your SDK and Kafka. For this, they use API Gateway as an HTTP interface to convert HTTP requests that will communicate with the Producer that has the SDK and can publish to Kafka.

Event Gateway

That way, we go through the same validation against the schema in the registry just as if any of our client code is using the producer SDK. This allows external 3rd parties to publish events without using our SDK directly. We can instead have them use our Event Gateway (HTTP API).

Technical Blog Posts

I love when companies have technical blog posts that give insights into their architecture and design. It’s hard to know the full context, but seeing how they solve these issues they run into is interesting. Companies face many common issues when using Event-Driven Architecture, but all have unique constraints.

If you have any recommendations for other technical blog post analyses, please let a comment!

Join!

Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

How your “Sr.” Devs incurred Technical Debt

Are you overwhelmed by technical debt? Taking the path of least resistance when implementing new features in a large existing codebase will ultimately turn it into a difficult-to-change turd pile. It’s a vicious circle. Making the “quick change” constantly makes it harder to make future changes. So what’s the solution? Being aware of technical debt, stop solely thinking about data, and give yourself options in your architecture.

YouTube

Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.

Path of Least Resistance

One common reason for a system growing over time and becoming unmaintainable is developers choosing to take the path of least resistance when implementing a change.

This happens for various reasons, such as time constraints, unfamiliarity with the system, lack of domain knowledge, poor overall architecture & design, etc.

For example, let’s say we have a typical web application that is using some underlying web framework that invokes some code into our application logic, through to our domain, and then some interaction with a database.

Application Request

When a new feature is implemented, it’s common to look at other features as templates for developing a new feature. Or, worse, it can be using an existing feature and adding the relevant code needed for the new feature throughout the stack. I say worse because this can often confuse two concepts that seem similar but are very distinct. Merging the two concepts within the same code path can add complexity.

Layers

This means we may change existing code through the entire stack, from the client, web API, application code, domain, and our database.

You may decide to piggyback off another feature because of time constraints. It’s not because the feature is difficult to implement. It’s time-consuming or will take more time than you have to implement. Or if you’re new to the codebase or it’s brittle, you might be afraid to make changes because you know it it can cause you to break other parts of the system and don’t want to cause any regressions.

The path of least resistance is making a change that you know isn’t going to break anything that isn’t overly time-consuming, but it’s not necessarily the ideal. It’s likely good for the right now but not good for the long run.

Technical Debt

Technical debt isn’t inherently bad. For me, technical debt comes in two forms. The first is when you’re aware and choosing to take on technical debt at a very moment, knowing it adds value now but will cause issues in the future. This awareness of choosing to make this explicit decision isn’t bad.

However, when you’re unaware that you’re making these types of decisions is when you’re headed in the wrong direction.

If you’re making explicit decisions about the tradeoffs of technical debt, you’re aware of the debt being incurred. You can then explicitly choose when to pay off (refactor) that debt. For example, with a startup, you might incur debt right now so that you have a future.

On the other side, if you’re unaware that you’re incurring technical debt, then when would you realize all the debt that’s been incurred and needs to be addressed? Taking the path of least resistance, without realizing it, is one form of this happening. While it seems like it’s helping you now, it could be hindering you now and even more so in the future.

Coupling & Cohesion

Software Architecture is about making key decisions at a low cost that give you options in the future. Having a good architecture allows you to evolve your system over time. As a codebase and system grow, it should not hinder future development. I’ve talked about this more in my post What is Software Architecture?

Why is a system brittle and hard to change? Generally, it has a high degree of coupling from higher and lower levels within a system. I find this is often because of the focus on data and informational cohesion rather than functional cohesion.

For example, let’s say we are in an e-commerce and warehouse system. There is the concept of a product. When we primarily think about data first, we think of a singular product. It holds all information for everything related to an individual product. The name, price, location in the warehouse, the quantity on hand, it is available for sale, etc.

In reality, a system for e-commerce and a warehouse would be huge. A large codebase that multiple departments would use in an organization. Sales, Purchasing, Warehouse (shipping & receiving), Accounting, and more.

In other words, I’m simplifying this example only to show a few different pieces of data related to a product, but in reality, there would be a lot.

When focusing on data primarily, we lose sight of the behaviors that relate to this data. What does the QuantityOnHand have to do with the Price? What does the Location have to do with the Description?

Nothing.

We’ve lumped all aspects into one concept of a product. However, in a large system like this, the concept of a product would exist in many different forms depending on the behaviors provided.

Product Entities

Sales have the concept of the product that cares about the Selling Price and if we’re selling. It’s customer focused.

Purchasing cares about the price from the vendor or manufacturer, which is our cost. It’s vendor-centric.

The warehouse cares about the location of the product in the warehouse and the assumed quantity on hand.

Each logical boundary has a concept of a product but has different concerns in each of its own contexts.

This means instead of mixing all these different concerns up together, instead be driven by the capabilities of each boundary and then the data ownership for those capabilities.

Low functional cohesion will lead to a high degree of coupling.

Defining logical boundaries by grouping related behaviors will lead to higher cohesion, which can then lead to loose coupling.

Awareness

Some of the trade-offs of taking the path of least resistance is being aware of the trade-offs you are making between coupling and cohesion. Earlier I mentioned piggybacking off an existing feature to implement a new feature. You’re coupling. Again, not a bad thing if that decision is explicit.

Over time, left unchecked, if you’re unaware of the technical debt you’re creating, you’ll end up with a large turd pile that’s brittle and hard to change.

If you are aware you can choose when to pay down debt (refactor) and keep making those decisions over time, you can manage the amount of debt incurred, never letting it get out of reach.

I often say a system is a turd pile because nothing is perfect. It’s a constant battle to pay down debt, whether you choose it explicitly or not.

Join!

Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design