Message-IDs for Handling Concurrency

This post serves as a guide for how you can use a Message identification (Message-IDs) on your messages (events and commands) to handle concurrency.

This post is in a series related to messaging. The overview is available in my Message Properties post.

Message-IDs

Each message, regardless of it being an event or a command, should contain a way to identify its specific instance of that message. This is as simple as adding a GUID/UUID to your messages:

No other message (event/command) should ever use this ID (unless you’re also using a message owner). It’s a one time only usage that should be unique.

The producer of the event should be creating this ID before it publishes it. It shouldn’t be hydrated by some middle party.

Concurrency

Most systems support at least once messaging. Meaning, they will deliver a message to the consumer at least once. This means it can be delivered more than once.

Message handlers should be reentrant. You should be able to invoke multiple instances of a message handler concurrently and safely.

Meaning if the exact same instance of an InventoryAdjusted event was invoked twice (with the same ID), at the same time, the outcome would be the inventory on hand would increase by 10, not 20.

We can achieve this by having our message handler use the message ID apart of the same transaction it’s using apart of its state change.

Here’s an example using pseudo C#

The example above is using a unique key on Handler and MessageID columns in the Concurrency table. When we either try to insert the record or commit the transaction, it will fail if the unique constraint fails.

Events

Since an event can have multiple consumers/handlers is the reason why I’ve included the name of the handler as apart of the unique constraint. This is to demonstrate if you were using the same concurrency table for multiple handlers. In the case of event handlers, you want each handler to process the event, but not more than once each.

If you were using this for commands, you wouldn’t need to record the handler, just the MessageID.

Related Links

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.

Roundup #65: CASPaxos, HealthChecks & Serilog, Fallback Policies, Playwright, F# Path to Relaxation

Here are the things that caught my eye recently in .NET.  I’d love to hear what you found most interesting this week.  Let me know in the comments or on Twitter.

CASPaxos: Linearizable databases without logs

Recently I’ve been playing around with a new algorithm known as CASPaxos. In this post I’m going to talk about the algorithm and its potential benefits for distributed databases, particularly key-value stores.

Link: https://reubenbond.github.io/posts/caspaxos

Excluding health check endpoints from Serilog request logging

In this post I show how to skip adding the summary log message completely for specific requests. This can be useful when you have an endpoint that is hit a lot, where logging every request is of little value.

Link: https://andrewlock.net/using-serilog-aspnetcore-in-asp-net-core-3-excluding-health-check-endpoints-from-serilog-request-logging/

Globally Require Authenticated Users By Default Using Fallback Policies in ASP.NET Core

You can use Fallback Policies in ASP.NET Core 3.0+ to require an Authenticated User by default. Conceptually, you can think of this as adding an [Authorize] attribute by default to every single Controller and Razor Page ONLY WHEN no other attribute is specified on a Controller or Razor Page like [AllowAnonymous] or [Authorize(PolicyName="PolicyName")]).

Link: https://scottsauber.com/2020/01/20/globally-require-authenticated-users-by-default-using-fallback-policies-in-asp-net-core/

Playwright

Playwright is a Node library to automate the Chromium, WebKit and Firefox browsers. This includes support for the new Microsoft Edge browser, which is based on Chromium.

Link: https://github.com/microsoft/playwright/blob/master/README.md

The F# Path to Relaxation – and what it means for .NET

After all the talk this week about .NET and it’s liveliness, I recommend watching this talk, re: memetic independence

Link: https://www.youtube.com/watch?v=ZTbyKsw7uIU

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.

Event Versioning Guidelines

Event Versioning Guidelines

This post serves as a guideline for how to handle versioning for messages, specifically in most cases, events. This Event Versioning Guidelines post is in a series related to messaging. You can find the overview in my Message Properties post.

Backward Compatible

Generally, adding additional properties to an event will not cause a versioning conflict with the event consumer.

Meaning if we have a contract that defines the shape of our event, as long as we don’t break that contract, the existing event consumers will be able to process our new event and will ignore the additional properties.

Also, if you are persisting events to an event/data store, you must be able to upcast an old stored event to our new version.

This isn’t that crazy of a concept. If you think about a relational database, when you need to add a new column, you generally will create the new column as having a default NULL value. The difference is you won’t backfill the existing events by replacing NULL with an actual value. Rather you handle the NULL value in your application code.

If you must change the contract, then you must create a new version of that event and cause a breaking change.

Version Number

One option for defining a new event is to include a version number within the event itself. Event consumers can then use this version number to determine how they should handle or process the event.

Regardless of backward compatibility, you can use a version number alongside your event name to indicate to consumers which contract to an event they are processing.

As an example using Semantic Versioning to define our events:

New Event

Most of the time if an event is not backward compatible, you likely have an entirely new event. And because of this, the name of the new event is likely different from the old event.

What’s tricky is to determine if this new event replaces the old event. Is the new event a discovery of an actual business event or is it simply a new understanding that replaces an existing concept?

Double Publish

If an event isn’t backward compatible, you might want to publish both v1 and v2 of the event that occurred.

Meaning the publisher writes both versions (v1 and v2) to the message queue or event stream.

One issue with this approach is consumers must understand this. They must understand they should only process one or the other. They either must process v1 and ignore v2, or process v2 and ignore v1.

This isn’t that big of a deal when the consumers aren’t replaying (re-processing) existing events from an event store. If you are using an event store and replaying events for a projection, you likely do not want to double publish. The reason being is processing both could have a negative impact.

If you’re not replaying events, then you’re consumers have to implement the new version you’re publishing. At the same time, they can remove the handling of the old event.

It’s not that odd to double publish by creating a new event and deprecate an old event and remove it once all consumers have handled the new event.

Event Versioning Guidelines

If you have any additional event versioning guidelines that I have omitted, please let me know on Twitter or in the comments.

Related Links

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.