Sponsor: Do you build complex software systems? See how NServiceBus makes it easier to design, build, and manage software systems that use message queues to achieve loose coupling. Get started for free.
If you’re thinking of building or already are implementing a system using async messaging (SOA or Microservices) then you need to start thinking about what type of messaging library you want to use in front of a message broker. CAP is an Event Bus that implements the Outbox Pattern to deal with distributed transactions.
YouTube
I did a live stream exploring CAP that is over on my YouTube Channel.
Distributed Transaction
When you’re using messages (events) to communicate between systems you will run into the situation where you need to save data to your database, then publish an event to a message broker. These events will then be received up by other systems for their own internal use.
The problem arises when two situations occur:
- You save data to your database but there is a failure when publishing the event to your message broker
- You publish an event to your message broker that something occurred in your system, but then when trying to save to your database, it fails.
In both situations, there is no consistency between what your database has saved and what you have published to the message broker.
What you want in this situation is one atomic transaction that can save data to your database and publish the event. If either fails, the other is rolled back. Basically, a distributed transaction.
Outbox Pattern
The Outbox pattern solves this issue by using a single transaction to perform both actions. What this involves is rather than publishing the event directly to the message broker, it serializes the event and saves it to a table in the same database using the same transaction for persisting your application data.
This is a diagram of the outbox pattern described in Microsoft’s eShopOnContainers reference application
Once the event is persisted to the table within the database, it will then be published to the message broker. If the message broker is unavailable or there is a failure publish to the message broker, the library you’re using will retry to publish. It provides reliability that you are not losing messages that you need to publish.
CAP
I stumbled upon the CAP project as I was looking for references for how other libraries have implemented the outbox pattern.
The following just illustrates the simplicity of using the CAP library and it’s API. My example uses MySQL as the database and RabbitMQ as the message broker.
First step is to include the relevant packages to your project
Next is to configure the Startup. In the ConfigureServices we need to use the AppCap() with various options to configure our database and message broker connections. Also in the Configure I’ve added the UseCapDashboard() which provides a little web-based UI for showing the messages and event subscriptions.
Now that we have CAP configured, the first step is going to be publishing an event. You do this by using the ICapPublisher. The parameters are a name and the contents of the event.
The key thing to point out here is that BeginTransaction() is an extension method from CAP. This extension method starts the MySQL transaction but also passes it along to the CapPublisher so they are using the same transaction. CapPublisher needs this transaction because it is going to write the published events to a table within your database.
CAP automatically creates this table. There is no setup required on your end.
Finally, we can create a subscriber/receiver for this event. To do so is simply a matter of implementing ICapSubscribe and add the CapSubscribe attribute to a method with the appropriate arguments that match what was published.
That’s it!
That’s the simplest example. And it’s pretty simple and hopefully gives you an idea of how CAP works with implementing the outbox pattern. There are various message brokers it supports such as Kafka, RabbitMQ, and Azure Service Bus. On the database side it supports SQL Server, MySQL, PostgreSQL, and MongoDB.
My entire sample application is available on GitHub.