Testing your Domain when Event Sourcing

How do you test your domain when Event Sourcing? I find testing aggregates with Event Sourcing to be simpler than testing if you’re storing the current state. The inputs to your aggregate are events and the output of your aggregate are events.

Given a stream of events
When a valid command is performed
Then new event(s) occurs

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything that is in this post.

Event Sourcing

I’ve covered Event Sourcing in my Event Sourcing Example & Explained in plain English post. So check that out if you need a more detailed primer.

Generally, I use Aggregates as a way to model my domain. The aggregate is what exposes commands and if those commands are called/invoked will result in creating new events. This general concept is how testing in event sourcing becomes simple.

In order to use an aggregate, you first need to pull all the existing events from the Event Store, replay them within the Aggregate to get to the current state, then return the Aggregate to the caller. To do this, I generally use a Repository that will do this work and build the Aggregate.

To illustrate this, we have client code that is using the repository to get the Aggregate. The repository calls the Event Store to get the existing Events (if any) for this specific aggregate.

Testing your Domain when Event Sourcing

At this point the Repository will create an empty aggregate and replay all the events it received from the Event Store.

Testing your Domain when Event Sourcing

Once it has rebuilt the aggregate, it then returns it back to the Client. The client at this point will likely call various methods/commands on the aggregate.

This was the first stage of the process. I call this the rehydrating stage. You’re rehydrating your aggregate back to its current state with all the past events. Remember this first stage as the “Given” as we start testing in event sourcing.

Creating Events

The rest of the example is using a Warehouse Product, which is the concept of a product that is in a warehouse.

Now that the client code has an aggregate, it will likely perform one or more commands on the aggregate.

If the client calls ShipProduct() command on the aggregate, and the aggregate is in a valid state to do so and passes all invariants, then it will create a new event that it will keep internally within it.

If the client code then called another command, another event would be appended to the internal list.

This is the second stage of the process where we’ve created new events which are the result of calling commands on our aggregate. Remember this second stage as the “When” stage of testing with event sourcing.

Saving Events

The last step is taking the newly created events that are in the aggregate and persisting those to the event store.

Testing your Domain when Event Sourcing

This means the client code will call back to our Repository passing it the Aggreagate.

Testing your Domain when Event Sourcing

The repository will get the new events and append those to the Event Store for that specific aggregates event stream.

Remember this stage as the “Then” in our tests when Event Sourcing.

Given, When, Then

If you take that basic 3 steps of loading an aggregate, calling commands, saving the new events, you can boil that down to:

Given a stream of events
When a valid command is performed
Then new event(s) occurs

You can use this as the testing strategy for testing your Aggregates.

The WarehouseProduct above has 3 commands: ShipProduct, ReceiveProduct, and AdjustInventory. All of which result in creating their respective events if they passed any invariants.

To illustrate this Given, When, Then for the ShipProduct command, which should create a ProductShipped Event.

While that does satisfy our goal, it’s a bit cumbersome to have to write this to verify the event. To simply and make this a bit more natural to read, I created a base class to use within our tests.

Now using this abstract class, here are all the tests for the WarehouseProduct.

Source Code

Developer-level members of my CodeOpinion YouTube channel get access to the full source for any working demo application that I post on my blog or YouTube. Check out the membership for more info.

Follow @CodeOpinion on Twitter

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.

Problem Details for Better REST HTTP API Errors

How do you tell your API Consumers explicitly if there are errors or problems with their request? Everyone creating HTTP APIs seems to implement error responses differently. Wouldn’t it be great if HTTP API Errors had a standard? Well, there is! It’s called Problem Details (https://tools.ietf.org/html/rfc7807)

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything that is in this post.

If you’re creating a RESTful HTTP API, Status Codes can be a good way to indicate to the client if a request was successful or not. The most common usage is 200 range status codes indicate a successful response and using 400 range indicate a client errors.

However, in most client error situations, how do you tell the client specifically what was wrong? Sure an HTTP 400 status code tells the client that there’s an issue, but what exactly is the issue?

Different Responses

Here are two made-up examples that were inspired by 2 different APIs that I’ve recently consumed. Both indicate there a client error, however, they both do this in very different ways.

This first example is using a Status Code of 400 Bad Request, which is good. They provide a response body that has a Message member which is human readable. There is also a documentation member which is a URI, I assume to give the developer more info on why it occurred.

Here’s another example but very different response.

This response has an HTTP 200 OK. Which is interesting, to say the least. Instead, to indicate success or failure, they include in the response body a success member was a Boolean. There is an error object which is useful because it contains an info member, which is human readable. But what’s really nice is the code and type members which appear to be machine-readable. Meaning we can read their documentation, and write the appropriate code to handle when we receive an error with code=101, then we might want to show our end-user some specific message or possibly perform some other type of action.

Commonality

So what do these 2 different HTTP APIs have in common when it comes to providing the client with error information?

Nothing.

They are both providing widely different response body’s and using HTTP status codes completely differently.

This means that every time you’re consuming an HTTP API, you have to write 100% custom code to handle the various ways that errors are returned.

At the bare minimum, it would be nice if there was a consistent and standard way to define the human-readable error message. In one of the responses, this was called “message”, in the other, it was “error.info“.

Problem Details

Wouldn’t it be great if there was a standard for providing error info to your clients? There is! It’s called Problem Details (RFC7807)

   HTTP [RFC7230] status codes are sometimes not sufficient to convey
   enough information about an error to be helpful.  While humans behind
   Web browsers can be informed about the nature of the problem with an
   HTML [W3C.REC-html5-20141028] response body, non-human consumers of
   so-called "HTTP APIs" are usually not.

   This specification defines simple JSON [RFC7159] and XML
   [W3C.REC-xml-20081126] document formats to suit this purpose.  They
   are designed to be reused by HTTP APIs, which can identify distinct
   "problem types" specific to their needs.

   Thus, API clients can be informed of both the high-level error class
   (using the status code) and the finer-grained details of the problem
   (using one of these formats).

Here’s an example using Problem Details for the first example defined above.

Problem Details

type“: URI or relative path that defines what the problem is. In a similar way, as the first example had a “documentation” member, this is the intent of this member as well. It’s to allow the developer to understand the exact meaning of this error. However, this is meant to also be machine-readable. Meaning this URI should be stable and always represent the same error. This way we can write our client code to handle this specific type of error how we see fit. It’s acting in a similar way as an error code or error key.

title“: A short human-readable message of the problem type. It should NOT change from occurrence to occurrence.

status“: The status member represents the same HTTP status code.

detail“: Human-readable explanation of the exact issue that occurred. This can differ from occurrence to occurrence.

instance“: A URI that represents the specific occurrence of the problem.

Here’s another example using Problem Details from the second example above.

Problem Details

Type & Extensions

In the example above, traceId is an extension. You can add any members you want to extend the response object. This allows you to provide more details to your clients when errors occur.

This is important because if you use the type member, which is the primary way you identify what the problem is, then you can provide more extension members based on the type you return.

In other words, in your HTTP API documentation, you can specify a problem type by its URI, and let the developer know there will be certain other extension members available to them for that specific problem.

Multiple Problems

As with everything, nothing is perfect. Problem Details has no explicit way of defining multiple problems in a single response. You can achieve this by defining a specific type, which indicates there will be a problems member which will be an array of the normal problem details members. Just as I described above to leverage bot the type and extensions together.

ASP.NET Core

If you’re using ASP.NET Core, you can use Problem Details today in your Controllers. Simply call the Problem() which returns an IActionResult.

If you don’t have thin controllers and have business logic outside of your controllers, then you can use Hellang.Middleware.ProblemDetails by Kristian Hellang which is a middleware that maps exceptions to problem details.

Source Code

Developer-level members of my CodeOpinion YouTube channel get access to the full source for any working demo application that I post on my blog or YouTube. Check out the membership for more info.

Follow @CodeOpinion on Twitter

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.

Event Based Architecture: What do you mean by EVENT?

The term “Event” is really overloaded. There are many different utilities that leverage events. Event Sourcing, Event Carried State Transfer, and Event Notifications. None of these are for the same purpose. When talking about an Event Based architecture, realize which one you’re using and for what purpose.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything that is in this post.

Overloaded

With the popularity of Microservices, Event Driven Architecture, Event Sourcing, and tooling, the term “Events” has become pretty overloaded and I find has been causing some confusion.

There’s a lot of content available online through blogs or videos that are often using the term Event incorrectly or in an ambiguous way. Almost every time I see the term event being used incorrectly is when the topic being covered is Microservices or Event Driven Architecture.

Here are the three different ways that the term “Event” is being used and in what context or pattern and for what purpose.

Event Sourcing

Event Sourcing is a different approach to storing data. Instead of storing the current state, you’re instead going to be storing events. Events represent the state transitions of things that have occurred in your system. Events are facts.

To illustrate the exact same product of SKU ABC123 that had a current state quantity of 59, this is how we would be storing this data using event sourcing.

This means that your events are the point of truth and you can derive current state from them.

The confusion when talking about Event Based Architecture comes in because I most often see people refer to Event Sourcing in a Microservices architecture as a means to communicate between services. Microservices and event sourcing are orthogonal. You do not need to be event sourcing in order to have a Microservice. Event sourcing has nothing to do with communication to other services. Again, event sourcing is about how you store data.

The reason, I believe, that there’s confusion is because when you’re event sourcing, you might be tempted to expose those events to other services. I say tempted because the events you’re storing as facts are not something you directly want to expose to other services. These events are internal and not used for integration.

Also, depending on the database or event store you’re using, it may also act as a message broker. This is often used within a service/boundary in order to create projections or read models from your event stream.

For more on projections, check out my post Projections in Event Sourcing: Build ANY model you want!

Event Carried State Transfer

The most common way I see events being used and explained is for state propagation. Meaning, you’re publishing events about state changes within a service, so other services (consumers) can keep a local cache copy of the data.

This is often referred to as Event Carried State Transfer.

The reason services will want a local cache copy of another service’s data, is so they do not need to make RPC calls to other services to get data. The issue with making the RPC call is if there are issues with availability or latency, the call might fail. In order to be available when other services are unavailable, they want the data they need locally.

In the example above, Warehouse and Billing require the Sales service. If the Sales service is unavailable, they may also be unavailable. To alleviate this, if they have the relevant data they need locally, it prevents them from having to make RPC calls to Sales.

What this looks like in practice is to use fat messages that generally contain all the data related to an entity.

Sales will publish a ProductChanged event that both the Warehouse and Billing will consume to update their local cache copies of a Product.

The contents of ProductChanged will generally look something like this:

The event will represent the entire current state of the entity. Meaning these events can get pretty large depending on the size of the entity.

While this approach is often used, I’d argue in most places if you’re doing this, you probably have some boundaries that are wrong. For more on defining boundaries check out my post on Defining Service Boundaries by Splitting Entities

Events as Notifications

The reason why Event Carried State Transfer is so popular when discussing Event Based Architecture is that Events will be used for notification purposes to other services, however being used incorrectly.

Most times events used for notifications are generally pretty slim. They don’t contain much data. If a consumer is handling an event but needs more information, to, for example, react and perform some action, it might have to make an RPC call back to the producing service to get more information. And this is what leads people to Event carried State Transfer, so they do not have to make these RPC calls.

To illustrate, the Sales service publishes an OrderPlaced event.

Event Based Architecture

The Billing Service is consuming this event so it can then create an Invoice.

Event Based Architecture

But because the event doesn’t contain much information, the Billing service then needs to make an RPC call back to Sales to get more data.

And this is how people then land on using Event carried State transfer to avoid this pattern.

Again, I sound like a broken record. But the likely reason this is occurring is because of incorrect boundaries. Services should own the data that relates to the capabilities they perform.

If you do have boundaries correct and each service has the relevant data for its capabilities, this means that events are used as a part of a workflow or long-running process. Events are used as notifications to tell other services that something has occurred.

To illustrate this again, Sales is publishing an OrderPlaced event that Billing is consuming.

Since Billing has all the data it needs, it creates an Invoice and publishes an OrderBilled event.

Event Based Architecture

Next, the Warehouse service will consume the OrderBilled event so that it can create a ShippingLabel for the order to be shipped.

Event Based Architecture

Once the shipping label has been created by the Warehouse, it publishes a LabelCreated event.

Finally, Sales is consuming the LabelCreated event so that it can update it’s Order status to show that the Order has been billed and is ready to be shipped.

This is called Event Choreography and is driven by events that are used for notifications. For more info check out my post Event Choreography & Orchestration (Sagas)

Event Based Architecture

Hopefully, this clears up some confusion about how the term even is used in different situations around event based architecture. I also hope it illustrates why Event Carried Transfer exists and the problem it’s trying to solve. However, that problem may likely be caused by incorrect boundaries.

Source Code

Developer-level members of my CodeOpinion YouTube channel get access to the full source for any working demo application that I post on my blog or YouTube. Check out the membership for more info.

Follow @CodeOpinion on Twitter

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.