Moving work Out-of-Process using Brighter and RabbitMQ

Once you start doing in-process request & event dispatching, you’ll soon want to move out-of-process so you can isolate work from the caller/invoker. This is often times the next logical step if you’re using MediatR for commands and especially for events/notifications. Here’s how you can do use the same paradigm for in-process and out-of-process using Brighter and RabbitMQ.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing you how to move work out-of-process.

MediatR

For those unfamiliar with MediatR library or the mediator pattern:

In software engineering, the mediator pattern defines an object that encapsulates how a set of objects interact. This pattern is considered to be a behavioral pattern due to the way it can alter the program’s running behavior.

I covered MediatR and why you might want to consider it but also one big drawback in my why use MediatR post.

One of the biggest downside to in-process processing (which MediatR does), is it does isolate the actual handling of a request. This isn’t a problem with MediatR at all, just the nature of it executing in-process.

For example, in the case of ASP.NET Core, you’re tied to the originating HTTP request. If the request your processing throws an exception, that will get bubbled up to ASP.NET Core which will return an HTTP 500 to the client.

Brighter

Brighter is a command dispatcher and command processor. I’m using it for these examples because if you’re familiar with MediatR, Brighter will seem very similar in terms of the API. It can dispatch requests in-process as well as go out-of-process by using a Message Broker like RabbitMQ.

There’s a whole bunch Brighter can do, but for the purpose of this post, I just want to show you how you can move from In-Process to Out-of-Process rather seamlessly.

In my opinion, using the same paradigm with the same library can be incredibly beneficial. For example, if you’re using an in-process approach for ASP.NET Core routes, you can at a later time decide to move them out-of-process with some rather minor code changes.

Here’s an example of using Brighter in the exact same fashion as using MediatR. This is simply creating a PlaceOrder object and then using Brighters CommandProcessor to send that request. Brighter will then invoke the PlaceOrderHandler.Handle method and pass the PlaceOrder object you created.

If you’re familiar with MediatR, this is very very similar. And as mentioned, this is still in-process.

In the above example, because we’re throwing an InvalidOperationException, this will bubble up and cause ASP.NET Core to return the client an HTTP 500.

Out-of-Process

Moving work out-of-process means we’re going to execute the work that needs to be done for a request to an entirely different physical process than where the request originated. Because of this, the work will be done asynchronously.

Here are a couple of simple diagrams of how that looks.

Brighter

There are two processes, the HTTP Server using ASP.NET Core, and a Message Processor. Both of these are separate .NET Console Apps. The Contracts and Implementation projects are class libraries that are reference by both HTTP Server and Message processor.

When an HTTP Request comes to ASP.NET Core, it invokes the OrdersController and the PlaceOrder route action. Instead of doing the work required then, we will use Brighter to send a message to the message broker (RabbitMQ).

Once the message has been sent to RabbitMQ, ASP.NET will immediately continue and the client will immediately receive the HTTP 204 NoContent that we’re returning.

It will no longer get the InvalidOperationException and an HTTP 500 error. This is because we are not doing that work in-process.

Here’s how the message process works:

Brighter

Brighter is configured in the Message Processor to receive messages from RabbitMQ. Brighter will receive the PlaceOrder request and then execute the PlaceOrderHandler.Handle passing the PlaceOrder object.

The code change to move from in-process to out-of-process is changing one method.

Instead of calling Send() on the CommandProcessor, we’re calling Post().

That’s it.

Mind you, there’s some configuration required in the ASP.NET Core and the MessageProcessor projects. Here’s a sample of the ConfigureServices in the ASP.NET Core Startup class.

Retries & More

One of the nice things about running out-of-process is having the ability to retry on failures. This is pretty straight forward with Brighter by using a UsePolicy attribute on the handle method.

More

Brighter has a a lot of features that can be found in their docs.

I realize in this post I have not covered the reasons why or when you should move work out-of-process. I’ll be covering those topics more in future blog posts and videos.

The intent with this post was to display Brighter and how a library that handles both in-process and out-of-process can be very beneficial when you need to make the transition.

If you have any questions or comments, let me know in the comments, Twitter, or YouTube.

Follow @CodeOpinion on Twitter

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.

Why use MediatR? 3 reasons why and 1 reason not

Why use MediatR? 3 reasons why and 1 reason not

The MediatR library by Jimmy Bogard has become increasingly popular over recent years, and deservedly so. By its own definition, it’s a simple, unambitious mediator implementation in .NET. Why are so many developers using it? Why should you use MediatR? Here are 3 reasons why you should at least consider using it and one reason why shouldn’t.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this one regarding MediatR.

What is MediatR?

For those unfamiliar with MediatR library or the mediator pattern:

In software engineering, the mediator pattern defines an object that encapsulates how a set of objects interact. This pattern is considered to be a behavioral pattern due to the way it can alter the program’s running behavior.

Reason #1: Decoupling

Most examples I’ve seen using MediatR are with ASP.NET Core, but that doesn’t mean that’s the only framework where it’s useful. The point is decoupling your application code from the top-level framework code. Whatever code is actually in charge of executing your code.

Here’s a example using ASP.NET Core MVC

The point is creating a request object that you pass to MediatR which in turn invokes the correct Handler for that request object.

The PlaceOrderHandler has no references to any types or APIs in ASP.NET Core.

Reason #2: Application Requests

The reason why decoupling from top-level framework code such as ASP.NET Core can be important is to ask yourself: Is the application I’m creating an application or a web application?

There’s a difference between a web application and just an application.

An application can have many different inputs. Not all incoming requests are going to be via HTTP. There may be various ways that interact with your application.

For example, you could have recurring jobs that need to perform specific actions at a given time during the day. Or you may also have work that gets triggered from specific messages you’re picking up from a Message Broker.

There are a number of ways that you may want to invoke behaviors in your system that don’t initiate from an HTTP request.

In my example above with ASP.NET Core, we’re ultimately converting an HTTP request into an Application Request. That Application request is entirely decoupled from any specific top level framework and could be invoke from anywhere.

Using MediatR to create application requests to cross an integration boundary.

Reason #3: Request Pipelines

Once you start thinking about application requests, you can go deeper into creating a pipeline for those requests.

If you’re familiar with ASP.NET Core middleware, the purpose is to define a pipeline for an HTTP request.

You can create the same concept using MediatR behaviors. There are a few different ways to create them but here’s an example of a that implements the IPipelineBehavior

When you call mediator.Send() this behavior is called first prior to your primary handler (PlaceOrderHandler). The RequestHandlerDelegate which is a delegate to call PlaceOrderHandler. This allows us to do primary work before calling it or possibly even short-circuiting and not calling it at all, say if we had some validation issues.

This means you could have any number of pre-processors that can be run prior to the primary handler, or post-processors that can be run after.

Why use MediatR

This is incredibly powerful and allows you to separate different concerns related to a given application request.

Why Not?

The reason you may not want to use MediatR is simply that everything is done in-process. Meaning that wherever process is calling mediator.Send() is also the same process that is executing the relevant Handler for that request.

Udi Dahan the founder of NServiceBus posed this question on Twitter with my response.

Why use MediatR

With everything in-process, this becomes really apparent with Events/Notifications (MediatR calls them Notifications).

A notification can have zero or multiple handlers. However the execution of all the notification handlers are done all in the same process.

Why use MediatR

In the example above, if you publish a notification to MediatR, and there are 3 handlers for that notification, it will execute all 3. But what happens if one of them fails and throws an exception?

That exception will bubble up back to where you published the message via MediatR. How do you want to handle this? Can you re-publish that message again and have the handlers that succeeded run again? What if one of them was sending an email?

You want event/notification handlers to be run in independently in isolation. This is where out-of-process messaging comes in.

In this case, you might want to look at NServiceBus, Brighter, MassTransit, Rebus.

More

If you want more details on how MediatR works, I’ve talked and written about this in my Fat Controller CQRS Diet where I use MediatR as an example on how to decouple your application code from top-level framework code.

Follow @CodeOpinion on Twitter

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.

Thin vs Fat Integration Events

When notifying other parts of a system of state changes I recommend creating integration events (also referred to as notifications). They are really low coupling because the producer doesn’t care who the consumers are. They simply publish integration events to a message broker and go on their merry way.

Loosely Coupled Monolith

This blog post is apart of a series of posts I’ve created around Loosely Coupled Monoliths. Although not specific to a monolith as this blog post applies to many architectures including microservices.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts.

Fat Events

Events generally start out thin but over time get polluted with more and more data that consumers need. Take for example the following OrderPlaced event that is published from our Sales bounded-context.

It only contains the OrderId, which may not seem entirely useful for other parts of the system. Let’s say that another bounded context called Shipping needs the ShippingAddress. We then add that to our OrderPlaced event.

As time goes on, if we’re not careful, we may add more and more data to this event. The next thing you know you have a fat event.

Thin Events

The alternative I see the most often are thin events that contain, for the most part, just IDs. This then requires the consumer to request back to the publisher to get the relevant data.

The flow of this is demonstrated below.

Integration Events

An HTTP request is made to our HTTP server which passes our request to a particular context/service. It then publishes a message to a message broker.

Integration Events

The message is then distributed to another context/service which now has to make a synchronous call (over HTTP, gRPC, whatever) to the context/service that published the event in the first place.

This is undermining the loose coupling we are achieving through asynchronous messaging. If we must make a synchronous call (via HTTP, gRPC, whatever) back to the producer of the event, then we require the producer to be available for us to get the relevant data.

If the producer isn’t available, do we have a local cache? Can we even use the local cache? Do we need the most up to date information to proceed with how we want to react to the event?

Boundaries

As mentioned earlier in my example, there is a Shipping bounded context that needed the ShippingAddress in the OrderPlaced event. And since we don’t want to synchronously call the Sales service to get that data, what do we do?

My example of the OrderPlaced event is very is deliberate. In this particular case, the boundaries are wrong.

The Shipping bounded context should own the ShippingAddress and how it receives it. Not the Sales or through the OrderPlaced.

Client Generated IDs

The key is generating IDs where the workflow occurs. In this case, it may be the client. If we generate the OrderId at the client, then we can make the relevant requests to each bounded context in the workflow to add our shipping information and place the order.

The flow of this is demonstrated below:

Integration Events

The client makes a PUT request with the address as the body/payload to our shipping bounded context with the URI of /shipping/ec012569-14b4-4bcc-8922-96d8a283e204. This GUID in the URI is our OrderId.

Although we don’t even have an order yet, we’re associating the address we want to persist with this particular OrderId.

Next, the client makes a request to the Sales bounded context to place the order. Notice it’s using the same OrderId in the URI.

Now when our OrderPlaced event is published and picked up by the Shipping bounded context, it already has the shipping information. It does not need it in the OrderPlaced event, nor does it need to make a call to the Sales bounded context to get it.

Event-Carried State Transfer

Context is king. Although I prefer thin events with well defined boundaries, some situations you may prefer to use fat integration events. None are bigger than Event-Carried State Transfer events. These events basically contain the entire state (at time of publish).

This is useful for having local caches of other bounded-context/service data. This reduces the need to make asynchronous calls to the other services but at the same time involves more complexity as you are maintaining your own copy of the state locally.

Questions or Feedback

If you have any questions or comments, let me know in the comments of the YouTube video, in the comments on this post or on Twitter.

Follow @CodeOpinion on Twitter

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.