Category: Architecture

Event Versioning Guidelines

Derek Comartin Leave a comment

Event Versioning Guidelines

This post serves as a guideline for how to handle versioning for messages, specifically in most cases, events. This Event Versioning Guidelines post is in a series related to messaging. You can find the overview in my Message Properties post.

Backward Compatible

Generally, adding additional properties to an event will not cause a versioning conflict with the event consumer.

Meaning if we have a contract that defines the shape of our event, as long as we don’t break that contract, the existing event consumers will be able to process our new event and will ignore the additional properties.

Also, if you are persisting events to an event/data store, you must be able to upcast an old stored event to our new version.

This isn’t that crazy of a concept. If you think about a relational database, when you need to add a new column, you generally will create the new column as having a default NULL value. The difference is you won’t backfill the existing events by replacing NULL with an actual value. Rather you handle the NULL value in your application code.

If you must change the contract, then you must create a new version of that event and cause a breaking change.

Version Number

One option for defining a new event is to include a version number within the event itself. Event consumers can then use this version number to determine how they should handle or process the event.

Regardless of backward compatibility, you can use a version number alongside your event name to indicate to consumers which contract to an event they are processing.

As an example using Semantic Versioning to define our events:

New Event

Most of the time if an event is not backward compatible, you likely have an entirely new event. And because of this, the name of the new event is likely different from the old event.

What’s tricky is to determine if this new event replaces the old event. Is the new event a discovery of an actual business event or is it simply a new understanding that replaces an existing concept?

Double Publish

If an event isn’t backward compatible, you might want to publish both v1 and v2 of the event that occurred.

Meaning the publisher writes both versions (v1 and v2) to the message queue or event stream.

One issue with this approach is consumers must understand this. They must understand they should only process one or the other. They either must process v1 and ignore v2, or process v2 and ignore v1.

This isn’t that big of a deal when the consumers aren’t replaying (re-processing) existing events from an event store. If you are using an event store and replaying events for a projection, you likely do not want to double publish. The reason being is processing both could have a negative impact.

If you’re not replaying events, then you’re consumers have to implement the new version you’re publishing. At the same time, they can remove the handling of the old event.

It’s not that odd to double publish by creating a new event and deprecate an old event and remove it once all consumers have handled the new event.

Event Versioning Guidelines

If you have any additional event versioning guidelines that I have omitted, please let me know on Twitter or in the comments.

Related Links

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.

Message Naming Guidelines

Derek Comartin Leave a comment
Message Naming Guidelines

As with many things in software development, naming can be difficult. The name of a message can be incredibly important which is why naming of your messages should be taken with consideration. This post is to be used for message naming guidelines with dos and don’ts based on my prior experiences of the good, the bad, and the ugly.

This blog post is in a series related to messaging.  You can find the overview in my Message Properties post.

YouTube

If you haven’t already, check out my YouTube channel where I’m covering this blog post in video form.

Commands vs Events

Commands and Events are the types of messages I generally deal with within a system. We need to differentiate between commands and events because they will be named differently depending on which one they are.

Commands

Commands are messages with the intent to change the state of the system. They are actions to be performed that have side effects. They may be created by an end-user or another part of the system that could be automated. They are a way to execute behavior.

I try to avoid using Create, Update, Delete, Save prefix as the verb. Rather I try to focus on the business process and the intent of the user to initiate the state transition. This can be one of the verbs mentioned above, but try and dig deeper to reveal the actual intent.

My general guideline for naming is a verb and a noun. Basically the action against a thing.

Do

PlaceOrder
GenerateInvoice
ApplyDiscount

Don’t

Order
SaveInvoice
Discount

Events

Events are statements of fact. They are named in the past tense. They are named this way because they have already occurred. Events are things that have happened within a system. They are the result (not in method return sense) of an executed command.

I try to avoid Created, Updated, Deleted suffix as the verb. Trying to stay way from CRUD naming to get to the essence of what actually occurred from a business process view.

My general guideline for naming an event is a noun and a verb. As mentioned the verbs are past tense.

Do

OrderPlaced
OrderInvoiced
DiscountApplied

Don’t

Ordered
InvoiceCreated
Discounted

Suffix

I generally prefer using a suffix of Command or Event to your types/classes. Ultimately, my only guideline is to be consistent in doing so. If you choose to suffix, always do it, if not, don’t ever do it. Consistency is key.

Examples

PlaceOrder vs PlaceOrderCommand

OrderPlaced vs OrderPlacedEvent

Serialization

If you are persisting events to an event store, document store, or relational database, you’re likely serializing your event objects.

Consumers of those events will then pull those serialized event objects out of the store and try and deserialize them back to an event object you want it to be in the original type.

For example, in .NET you might be serializing using JSON.NET;

The issue now is how do you deserialize this string back to an OrderPlaced type?

Don’t

The first reaction may be to include the CLR Type name alongside the JSON. You can accomplish this by using the TypeNameHandling setting the serializing:

This would result in a $type property that points to the CLR type. When deserializing this string, you can now get it back as the original type.

I discourage using the explicit CLR type information alongside the event because you are very likely going to either rename, change the namespace, or assembly of an event. And at this point, you won’t be able to deserialize using this information.

Do

Include and persist a separate piece of static data alongside your serialized event to define what the name of the event is. This should be concrete and manually mapped from this string to a specific type you want to deserialize to. This allows you to move the specific event type to any assembly, namespace or even rename.

You can see an example of this in my SQLStreamStore demo/sample application.

Message Naming Guidelines

If you have any additional message naming guidelines that I have omitted, please let me know on Twitter or in the comments.

Related Links

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.

Message Properties

Derek Comartin Leave a comment
Message Properties

Over the years I’ve learned various aspects of messaging either through using various products, reading other blogs, or unfortunately in a lot of cases, discovering them myself. One of those aspects is message properties.

Specifically, (meta) data that you might want to include in your messages to solve some common problems.

My plan is to dive deeper with implementation examples for each topic listed in this post. Hopefully, this post can serve as a reference point to your adventures in messaging or possibly gain some insight that provides a solution for a pain point you may (already) be experiencing.

I was inspired to write this post after seeing this tweet:

Type/Name

The naming of messages is incredibly important for a variety of reasons. When talking about events, it’s important to name them in the past tense because it conveys that they are statements of fact.

Most often the name of an event is used to determine how to deserialize into a specific type. Where I’ve personally run into problems, in .NET specifically, is using the CLR type name as the event name to determine how to deserialize. The issue with this is you’re very likely to change the name of your CLR type.

Prefer to having a static name for an event and not using the CLR type name at runtime to create the event name.

More info on naming can be found in my post Message Naming Guidelines

Message Version

Most often when needing to version a message is to simply create a new type/name. With events, I think this approach works fairly well when you discover a better way to describe the event and along with it the data to expose.

When a new event isn’t required, but rather a slight change in data is to specify the version in the message. Consuming clients can use the message name along with the version to determine how to process the body of the message.

More on versioning can be found in my Event Versioning Guidelines

Message ID

A unique identifier of a message. This can be used for concurrency. Message Handlers can keep track of which messages IDs they have processed and determine if they should act upon receiving a message.

This is important because message handlers should be reentrant and messages can be delivered more than once.

More on this can be found on my Message-IDs for handling Concurrency post.

Correlation ID

A unique identifier that allows you to reference the flow of events (event chain) or a transaction. Also, often called a Trace ID. Assigning a correlation ID as soon as possible and any message handler that receives a message with a correlation ID then uses that same ID for any messages it outputs.

This is very useful for logging and error handling in asynchronous and distributed environments.

Contract Owner

Who owns the message contract can be important. Depending on how messages are distributed, there may be many messages from various owners. Knowing who owns a message contract is helpful for troubleshooting as well as how to handle that message.

An OrderPlaced event seems like it could be unique across all services, however, in a distribution domain, is that a Sales OrderPlaced from the Sales service or a Purch OrderPlaced from the purchasing service?

Having the owner of the message provides the ability to distinguish how to handle the message when the name alone doesn’t provide enough distinction.

UTC Timestamp

This seems obvious but it gives more context to when the message was created. A useful is keeping track of latency between a message being created to how long it’s taking a given service to process it.

Sender

Who originated the message. Not the service, but rather the user. This could be an access token that is then used to determine if the sender has the correct permissions or is authorized to perform the given request.

Resource/Entity Version

The version of the resource/entity that you are performing an action on. Similar to the message ID this can be used for concurrency. A wonderful example of this is Azure Cosmos DB which uses an ETag as the resource version which is then used with an If-None-Match header for optimistic concurrency. If you’re using Cosmos DB check out my post on Optimistic Concurrency in Azure Cosmos DB

Message Properties

I’ll be going in-depth in each of the above topics with code examples for the actual implementation of how you would use these message properties. Stay tuned.

If you’d like to add to the list, let me know in the comments or on Twitter.

Related

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.