Avoiding Batch Jobs by a message in the FUTURE

Some people will call it cron jobs, scheduled tasks, or batch jobs. Whatever you call it, it’s a process that runs periodically looking at the state of a database to determine some specific action to occur for the various records it finds. If you’ve dealt with this, you probably know it can be a nightmare, especially with failures. And of course, these usually run in the middle of the night, so get ready for a page!

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything in this post.

Reservation

A while back I placed an online order at a big box store that was set for pick-up at the store, rather than delivery. Here’s the confirmation order:

Order Confirmation Email

Note that the email says, once my order is ready for pick-up I’ll get an email. This is because an employee at the store has to physically go to get the item off the shelf somewhere in the store, and then bring it to a holding area for pick-up orders.

Once the item is taken from the shelf by the employee at the store, they set the order as being available for pick-up, which triggers this email I received:

Order Ready Email

The interesting part of this email is that it says I have 7 days to pick up my order, otherwise I’ll be refunded and the item will be put back onto the shelf.

Basically, this is a reservation. For more on the Reservation Pattern, check out my post on Avoiding Distributed Transactions with the Reservation Pattern.

Timeline

There are a couple of different scenarios. The first is that I actually drive to the store and pick up the item that I ordered.

The timeline would be that I placed my order, sometime later the order was reserved (taken from the shelf by an employee), and then later I arrived at the store to complete my reservation/order.

Reservation Completed Timeline

The second scenario is that I place my online order, it’s reserved for me, but I just never show up to pick the item up at the store.

Reservation Expired Timeline

After 7 days from the order being reserved, the reservation will expire, which means an employee will take the item and put it back on the shelf so someone else can purchase it.

Batch Jobs

If you’re using batch jobs, how would that work? Typically you’ll have a process that runs every day that will look at the state of the database and then determine what to do. In this example, this might be selecting all the orders that are reserved but have not yet been completed.

Batch jobs looking at the state of the database

At this point, the batch job has to do all of the refunds to credit cards and update the database to set the orders as canceled.

The issue with batch jobs is that they are done in a batch. If there are 100 orders that have expired, the batch job iterates one by one performing these actions. What happens if at order 42 of 100 the process fails, because of a bug or failing to connect to the payment gateway, etc.

Batch jobs failing

Batch jobs, purely based on the name or not done in isolation. This means the entire batch fails. Typically batch jobs are run during off-peak hours, which is usually in the middle of the night. If it fails in the middle of the night, you’re likely to get a page/alert of the failure.

Isolation

Rather what we would want is to handle each individual expired order/reservation in isolation. What we want to do is tell the system in the future, as a reminder, to cancel the reservation/order in 7 days when it’s reserved.

Expire Reservation Future Message

If the customer doesn’t come and pick up the item at the store, then the “expire reservation” reminder will kick in and execute at the appropriate time, which is 7 days from when the order was reserved.

However, if the customer does come and pick up the item at the store which completes the order, the “expire reservation” will still be triggered, however, it won’t need to do anything since the order is complete. It can exit early and not perform any action.

Reservation Completed Timeline

Each Order will have its own “expire reservation”. Each is executed exactly 7 days from when the order was reserved and each will be executed in isolation. If one fails, it doesn’t affect the others.

Delayed Delivery

So how can we technically implement this type of future reminder? You can use a queue that supports delayed delivery.

When sending a message to a queue, you’re also going to provide the queue with a period of time to delay the delivery to a consumer.

Send message to a Queue with Delayed Delivery

This means that the queue will not make the message visible to consumers. Consumers will not be able to pull this message from the queue until the delay period of time has elapsed. The consumers won’t even see the message in the queue.

Message not visible during period of time

Once the delayed delivery period has elapsed, the message will be visible again.

Message visible again

The consumer will then pull the message and process it.

Consumer processes message

What this allows us to do is send a message to be processed in the future.

In the example of our online order, this means that once an Order is reserved and the item is taken from the shelf, we enqueue a “expire reservation” message with delayed delivery of 7 days. After the 7 days, we will process that “expire reservation” message. If the order is already completed, the process just exits early. If it hasn’t been completed, it then does the credit card refund and updates the database to set the Order as canceled.

Avoiding Batch Jobs

You can avoid batch jobs and have smaller units of work in isolation by telling your system to do something in the future. Leveraging a queue that supports delayed delivery is one example of how you can accomplish this.

The workload will be smoothed out across time and provide isolation so you can handle failure at an individual level of work.

Join!

Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design as well as access to source code for any working demo application that I post on my blog or YouTube. Check out the YouTube Membership or Patreon for more info.

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

Sidecar Pattern for Abstracting Common Concerns

What is the sidecar pattern? Applications and services often have generic concerns such as health checks, configuration, metrics, as well as how they communicate with each other either directly or through messaging. Services usually implement these using libraries or SDKs to handle these concerns. How can you share these concerns across all these services so you’re not implementing them in every service? The sidecar pattern and ambassador pattern might be a good fit to solve this problem.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything in this post.

Shared Concerns

Regardless of the platform, you’re going to leverage libraries/packages/SDKs to handle common concerns like health checks, configuration, metrics, and more. Each service will have these common concerns and need to use the libraries for their respective platform.

For example, if you had two services, one written in .NET and the other written in Go. Each service would leverage libraries in its ecosystem to provide this common functionality.

Shared concerns for different services

Even with libraries, you still need to define how you’re handling and dealing with these common concerns. You may be using the same underlying infrastructure for both of them. As an example, each service might be publishing metrics to Cloudwatch.

Wouldn’t be nice if there was a standardized way that each service would handle these shared concerns?

Sidecar pattern

The sidecar pattern allows you to extract the common concerns from your service and host them separately in a separate process, known as a sidecar.

sidecar process runs locally to service

In the containerized world, this is often thought of as a separate container from your service, however, it really is just a separate process that runs locally alongside your service.

With a sidecar, your service can now interact with a separate process that will handle the common concerns. The sidecar can perform health checks on your service, your service can send metrics to the sidecar, etc.

Ambassador Pattern

Back to the example of sending metrics to Cloudwatch. With the sidecar, we can also apply the Ambassador Pattern. This all makes the sidecar a proxy to send data to Cloudwatch. This means our service is interacting with the sidecar, using a common API, and it’s in charge of sending the data to Cloudwatch. It’s not just metrics and Cloudwatch, this applies to any external service.

sidecar as a proxy to external services

The benefit is that the sidecar is handling any failures, retries, backoffs, etc. We don’t have to implement all kinds of retry logic in our service, rather that’s a common concern handled by the sidecar when communicating with external services.

If we want to make an HTTP request to an external service, we proxy it through the sidecar.

sidecar handling retry logic

If there is a transient failure, the sidecar is responsible for doing the retry while still maintaining the connection to our service while it does so.

Handling transient failures and retries

Abstraction

While each service has its own instance of a sidecar, you can be using the same sidecar for many different services. This allows each service to have the exact same interface to all of the shared concerns the sidecar provides.

This means that you can also be using it for things like a message broker.

sidecar as abstraction to message broker

If you read any of my other blog posts or watch my videos on YouTube, you know I’m an advocate for loosely coupling between services using messages, and not using blocking synchronous request-response.

A sidecar can provide a common abstraction over a message broker. This means that each service doesn’t have to interact with the broker directly, nor does it need to use the specific libraries or SDKs for that broker. The sidecar is providing a common API for sending and consuming messages and is abstracting the underlying message broker.

different services to AMQP supported message broker

This means you could have one service using .NET and another using Python, both exchanging messages to a broker supporting AMQP. Each service would be completely unaware of what that underlying transport and message broker is.

Trade-offs

So why would you want to use a sidecar and ambassador patterns? If you have services that are using different languages/platforms, and you want to standardize common concerns. Instead of each service implementing using their native packages/libraries/SDKs to their respective platform, the sidecar pattern allows you to define a common API that each service uses. This allows you to focus more on what your service actually provides rather than common infrastructure or concerns.

One trade-off to mention is latency. If you’re using a sidecar with the ambassador pattern to proxy requests to external services, a message broker, etc, you’re going to be adding latency. While this latency might not that much, it’s something to note.

If you don’t have a system that’s comprised of many services, or they are all using the same language/platform, then a sidecar could just be unneeded complexity. You could leverage an internal shared package/library that each service would use for defining a common API for shared concerns, it doesn’t need to be a sidecar. Again, your context matters. Use the patterns when you have the problems these patterns solve.

Join!

Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design as well as access to source code for any working demo application that I post on my blog or YouTube. Check out the YouTube Membership or Patreon for more info.

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

Microservices gets it WRONG defining Service Boundaries

Logical boundaries aren’t physical boundaries. A service boundary can be a one-to-one mapping of logical boundary and physical boundary but they don’t have to be. A logical boundary can have many physical boundaries and a physical boundary can be composed of many components from many logical boundaries. Do we have microservices to thank for the idea that the physical, development, and logical views are the same? This introduces a pile of unneeded complexity related to deployment, versioning, and data duplication.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything in this post.

Physical & Logical Boundaries

When most people think of service boundaries, they are often thinking of this diagram below. The dotted line represents a service boundary, in which both services communicate has their own database, however, they communicate directly with each other.

Microservices gets it WRONG defining Service Boundaries

Now if I were to specifically call out physical boundaries, I would have illustrated with a dotted line that the databases are a physical aspect separately from the service itself.

Microservices gets it WRONG defining Service Boundaries

This is pretty obvious as each service is probably making network calls to a database instance.

So now let’s throw some Client/UI into the mix. How would that fit into the diagram illustrating physical boundaries? It may look something like this:

However, in my experience, most folks aren’t building a dedicated UI/Client for each service, rather they often build a SPA, maybe a mobile app, etc.

Now I’m excluding an API gateway or BFF (Backend for frontend) in this diagram, but that could exist as well, in which the Client/UI would communicate with that, and it in turn would communicate with the various services.

What I’m really illustrating here is the idea that there is usually a frontend and a backend. The backend is represented by possibly many different services.

Based on the diagram, if I were to ask how many logical boundaries there are, you would likely say 3. If you are, it’s because you’re assuming my illustrating of logical boundaries is also represented in the same way as physical boundaries. The dotted lines represent physical boundaries, not logical boundaries. However, our industry has standardized physical boundaries being the same as logical boundaries. While they can be, they don’t have to be.

Service Boundaries

What’s a service? What are its boundaries?

A service is the authority of a set of business capabilities.

Behind those capabilities are the data that it owns. It’s a grouping of functionality that serves the business. Notice I have said nothing about technology? How you choose to develop and physically deploy those capabilities has nothing to do with defining a logical boundary.

So if we take the earlier diagram, and outline what the logical boundaries are, they are now vertical, not horizontal.

Microservices gets it WRONG defining Service Boundaries

A portion of the UI is owned by a local service boundary. The UI, the actual behaviors and capabilities, and the data all together define the logical boundary.

The issue is that there are multiple different ways to view a system. Below is a diagram that illustrates the 4+1 Architectural View Model by Philippe Kruchten

Without going into full detail about what each view represents, what I’ve been illustrating is that the logical view and physical views can be different. They do not have to be exactly the same.

If you were to think of a typical microservices implementation, the logical view and physical view are often the same. And if you’re not using a mono-repo, and have a git repo per each service, you then likely have the logical, development, and physical views all exactly the same.

A service boundary should own the entire vertical. This is why the UI is included when I’m illustrating the logical boundaries. The frontend isn’t its own logical service.

Microservices gets it WRONG defining Service Boundaries

This means that you can deploy your logical boundaries in many different ways. As an example of physical boundaries, you could have a UI that’s composed of multiple logical boundaries. The actual Service API could be composted together into a single process/container. A single database instance could hold the schema and data for each logical service boundary.

Microservices gets it WRONG defining Service Boundaries

So if we again jump back to the logical view, this turns into slicing everything vertically.

Microservices gets it WRONG defining Service Boundaries

Why does it matter?

So who cares? What’s the big deal if we treat physical boundaries as local boundaries? Unneeded complexity.

Boundaries are hard to define. When you focus on business capabilities and not technical concerns, you have to get deeper insights into the actual business domain. Understanding the capabilities that a service must provide and the data ownership behind those capabilities isn’t trivial to figure out.

If you treat local boundaries and physical boundaries as being the same thing, and you get these capabilities wrong (which you likely will), you’ll end up doing a lot of service-to-service communication, likely over a network via RPC. This path leads you to a distributed monolith, that is often worst than a traditional monolith. For more on this specifically, check out my post: REST APIs for Microservices? Beware!

A distributed monolith can be horrific because you’ll lose consistency if you really need it when performing commands. If you’re reaching out over RPC to another service boundary to get data to perform a command, the moment you get that data it’s inconsistent/stale. There is no distributed transaction.

Another issue is versioning and deployment. If you make a change to a service, but need all other services to make the appropriate changes as well, and they all need to get deployed together, you’re losing one of the primary benefits. You lost the ability to do independent deployments. Your services aren’t autonomous.

Logical boundaries do not need to be physical boundaries. They can be, but they do not have to be. You can have multiple services that are all composed and deployed in the same process. They could all live within the same git repo. Should you do this? Maybe that is actually best in your context. Maybe it’s not. But don’t get stuck thinking they’re the same thing.

Defining logical boundaries is hard. You’ll likely get it wrong. If you’re developing a new system and are unsure of where boundaries may lie, don’t put the added constraints of making your logical and physical boundaries the same as it will be much more difficult to change and manage.

Join!

Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design as well as access to source code for any working demo application that I post on my blog or YouTube. Check out the YouTube Membership or Patreon for more info.

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design