CAP: Event Bus & Outbox Pattern

Outbox Pattern

If you’re thinking of building or already are implementing a system using async messaging (SOA or Microservices) then you need to start thinking about what type of messaging library you want to use in front of a message broker. CAP is an Event Bus that implements the Outbox Pattern to deal with distributed transactions.

YouTube

I did a live stream exploring CAP that is over on my YouTube Channel.

Distributed Transaction

When you’re using messages (events) to communicate between systems you will run into the situation where you need to save data to your database, then publish an event to a message broker. These events will then be received up by other systems for their own internal use.

The problem arises when two situations occur:

  • You save data to your database but there is a failure when publishing the event to your message broker
  • You publish an event to your message broker that something occurred in your system, but then when trying to save to your database, it fails.

In both situations, there is no consistency between what your database has saved and what you have published to the message broker.

What you want in this situation is one atomic transaction that can save data to your database and publish the event. If either fails, the other is rolled back. Basically, a distributed transaction.

Outbox Pattern

The Outbox pattern solves this issue by using a single transaction to perform both actions. What this involves is rather than publishing the event directly to the message broker, it serializes the event and saves it to a table in the same database using the same transaction for persisting your application data.

This is a diagram of the outbox pattern described in Microsoft’s eShopOnContainers reference application

Outbox Pattern

Once the event is persisted to the table within the database, it will then be published to the message broker. If the message broker is unavailable or there is a failure publish to the message broker, the library you’re using will retry to publish. It provides reliability that you are not losing messages that you need to publish.

CAP

I stumbled upon the CAP project as I was looking for references for how other libraries have implemented the outbox pattern.

The following just illustrates the simplicity of using the CAP library and it’s API. My example uses MySQL as the database and RabbitMQ as the message broker.

First step is to include the relevant packages to your project

Next is to configure the Startup. In the ConfigureServices we need to use the AppCap() with various options to configure our database and message broker connections. Also in the Configure I’ve added the UseCapDashboard() which provides a little web-based UI for showing the messages and event subscriptions.

Now that we have CAP configured, the first step is going to be publishing an event. You do this by using the ICapPublisher. The parameters are a name and the contents of the event.

The key thing to point out here is that BeginTransaction() is an extension method from CAP. This extension method starts the MySQL transaction but also passes it along to the CapPublisher so they are using the same transaction. CapPublisher needs this transaction because it is going to write the published events to a table within your database.

CAP automatically creates this table. There is no setup required on your end.

Finally, we can create a subscriber/receiver for this event. To do so is simply a matter of implementing ICapSubscribe and add the CapSubscribe attribute to a method with the appropriate arguments that match what was published.

That’s it!

That’s the simplest example. And it’s pretty simple and hopefully gives you an idea of how CAP works with implementing the outbox pattern. There are various message brokers it supports such as Kafka, RabbitMQ, and Azure Service Bus. On the database side it supports SQL Server, MySQL, PostgreSQL, and MongoDB.

My entire sample application is available on GitHub.

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.

Configuring Errors and Warnings in C#

Configuring Errors and Warnings in C#

I recently stumbled upon some code that was making an awaitable method call but was not being awaited. I noticed the issue because it was a compiler warning CS4014 and was highlighted in code. I immediately fixed the issue, however, my second action was configuring errors and warnings in C#.

YouTube

Make sure to check out my YouTube channel where I often post related content that accompanies my blog posts.

WarningAsError

You can have a .NET build fail by having the compitler producing errors for normally what are considered warnings. Such as as CS4014

Because this call is not awaited, execution of the current method continues before the call is completed. Consider applying the ‘await’ operator to the result of the call.

The code in question looked something similar to this as I seen within Rider

There are very few cases where I would not want to await. Because of that, I wanted this to cause a compiler error so going forward it was impossible to do this by accident and not notice.

The fix is really simple, in your SDK style csproj file, specify an element called WarningAsError inside a PropertyGroup. The value is a comma-separated list of warning numbers to treat as errors.

Now when you look or build it produces an error instead of a warning.

The great thing about Roslyn analyzers is they are also helpful in finding possible code issues. One of my favorites is the Microsoft.VisualStudio.Threading.Analyzers that is available as a NuGet package. Simply add it to your project and you’ll immediately get over a dozen analyzer rules applied to your codebase. The package is best for detecting Sync over Async which I’ve written about before.

One of them is VSTHRD103 that finds any code that synchronously blocks. using Wait() on Task is a good example.

We can also turn this into an error by adding VSTHRD103 in our WarningAsErrors list.

NoWarn

However, when you start adding some Roslyn analyzers you may not like some of the warnings they provide. One for me, which is highly debated, is the use of the Async suffix. Since our entire codebase is Async, we do not follow this convention. This is rule VSTHRD200 which I did not want to be warned about. This is also pretty simple by adding a NoWarn element.

Directory.Build.props

The last tip is instead of specifying all of these in each project file (csproj) definition, you can create a file called Directory.Build.props and place it at the root of where your solution is located. This file will then apply to every project.

Configuring Errors and Warnings in C#

Do you have any preferred Roslyn analyzers or warnings you like to treat as errors? Let me know in the comments or on Twitter.

Links

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.

Roundup #72: Succinct C#, IHostedService Shutdown Timeout, try-convert, Coupling, Cohesion, and Microservices

Writing More Succinct C#

When looking at a lot of C# code nowadays, I find myself thinking “wow, that code could be made SO MUCH SMALLER!”. C# is a very flexible language, allowing you to write clean and functional code, but also very bloated code.

Link: https://www.danclarke.com/2020-more-succinct-csharp

Extending the shutdown timeout setting to ensure graceful IHostedService shutdown

I was seeing an issue recently where our application wasn’t running the StopAsync method in our IHostedService implementations when the app was shutting down. It turns out that this was due to some services taking too long to respond to the shutdown signal. In this post I show an example of the problem, discuss why it happens, and how to avoid it.

Link: https://andrewlock.net/extending-the-shutdown-timeout-setting-to-ensure-graceful-ihostedservice-shutdown/

try-convert

This is a simple tool that will help in migrating .NET Framework projects to .NET Core.

This tool is for anyone looking to get a little help migrating their projects to .NET Core (or .NET SDK-style projects).

As the name suggests, this tool is not guaranteed to fully convert a project into a 100% working state. The tool is conservative and does as good of a job as it can to ensure that a converted project can still be loaded into Visual Studio and build. However, there are an enormous amount of factors that can result in a project that may not load or build that this tool explicitly does not cover. These include:

Complex, custom builds that you may have in your solution

API usage that is incompatible with .NET Core

Unsupported project types (such as Xamarin, WebForms, or WCF projects)

If the bulk of your codebase is generally capable of moving to .NET Core (such as lots of class libraries with no platform-specific code), then this tool should help quite a bit.

Link: https://github.com/dotnet/try-convert

Build Stuff #6 e-Meetup – Sam Newman – Coupling, Cohesion, and Microservices

The terms coupling and cohesion come from the world of structured programming, but they are also thrown about in the context of microservices. In this session, I look at the applicability of these terms to a microservice architecture and also do a deep dive into the different types of coupling to explore how ideas from the 1970s still have a lot of relevance to the types of systems we build today.

Link: https://www.youtube.com/watch?v=s_bAC_X57MA

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.