Problem Details for Better REST HTTP API Errors

How do you tell your API Consumers explicitly if there are errors or problems with their request? Everyone creating HTTP APIs seems to implement error responses differently. Wouldn’t it be great if HTTP API Errors had a standard? Well, there is! It’s called Problem Details (https://tools.ietf.org/html/rfc7807)

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything that is in this post.

If you’re creating a RESTful HTTP API, Status Codes can be a good way to indicate to the client if a request was successful or not. The most common usage is 200 range status codes indicate a successful response and using 400 range indicate a client errors.

However, in most client error situations, how do you tell the client specifically what was wrong? Sure an HTTP 400 status code tells the client that there’s an issue, but what exactly is the issue?

Different Responses

Here are two made-up examples that were inspired by 2 different APIs that I’ve recently consumed. Both indicate there a client error, however, they both do this in very different ways.

This first example is using a Status Code of 400 Bad Request, which is good. They provide a response body that has a Message member which is human readable. There is also a documentation member which is a URI, I assume to give the developer more info on why it occurred.

Here’s another example but very different response.

This response has an HTTP 200 OK. Which is interesting, to say the least. Instead, to indicate success or failure, they include in the response body a success member was a Boolean. There is an error object which is useful because it contains an info member, which is human readable. But what’s really nice is the code and type members which appear to be machine-readable. Meaning we can read their documentation, and write the appropriate code to handle when we receive an error with code=101, then we might want to show our end-user some specific message or possibly perform some other type of action.

Commonality

So what do these 2 different HTTP APIs have in common when it comes to providing the client with error information?

Nothing.

They are both providing widely different response body’s and using HTTP status codes completely differently.

This means that every time you’re consuming an HTTP API, you have to write 100% custom code to handle the various ways that errors are returned.

At the bare minimum, it would be nice if there was a consistent and standard way to define the human-readable error message. In one of the responses, this was called “message”, in the other, it was “error.info“.

Problem Details

Wouldn’t it be great if there was a standard for providing error info to your clients? There is! It’s called Problem Details (RFC7807)

   HTTP [RFC7230] status codes are sometimes not sufficient to convey
   enough information about an error to be helpful.  While humans behind
   Web browsers can be informed about the nature of the problem with an
   HTML [W3C.REC-html5-20141028] response body, non-human consumers of
   so-called "HTTP APIs" are usually not.

   This specification defines simple JSON [RFC7159] and XML
   [W3C.REC-xml-20081126] document formats to suit this purpose.  They
   are designed to be reused by HTTP APIs, which can identify distinct
   "problem types" specific to their needs.

   Thus, API clients can be informed of both the high-level error class
   (using the status code) and the finer-grained details of the problem
   (using one of these formats).

Here’s an example using Problem Details for the first example defined above.

Problem Details

type“: URI or relative path that defines what the problem is. In a similar way, as the first example had a “documentation” member, this is the intent of this member as well. It’s to allow the developer to understand the exact meaning of this error. However, this is meant to also be machine-readable. Meaning this URI should be stable and always represent the same error. This way we can write our client code to handle this specific type of error how we see fit. It’s acting in a similar way as an error code or error key.

title“: A short human-readable message of the problem type. It should NOT change from occurrence to occurrence.

status“: The status member represents the same HTTP status code.

detail“: Human-readable explanation of the exact issue that occurred. This can differ from occurrence to occurrence.

instance“: A URI that represents the specific occurrence of the problem.

Here’s another example using Problem Details from the second example above.

Problem Details

Type & Extensions

In the example above, traceId is an extension. You can add any members you want to extend the response object. This allows you to provide more details to your clients when errors occur.

This is important because if you use the type member, which is the primary way you identify what the problem is, then you can provide more extension members based on the type you return.

In other words, in your HTTP API documentation, you can specify a problem type by its URI, and let the developer know there will be certain other extension members available to them for that specific problem.

Multiple Problems

As with everything, nothing is perfect. Problem Details has no explicit way of defining multiple problems in a single response. You can achieve this by defining a specific type, which indicates there will be a problems member which will be an array of the normal problem details members. Just as I described above to leverage bot the type and extensions together.

ASP.NET Core

If you’re using ASP.NET Core, you can use Problem Details today in your Controllers. Simply call the Problem() which returns an IActionResult.

If you don’t have thin controllers and have business logic outside of your controllers, then you can use Hellang.Middleware.ProblemDetails by Kristian Hellang which is a middleware that maps exceptions to problem details.

Source Code

Developer-level members of my CodeOpinion YouTube channel get access to the full source for any working demo application that I post on my blog or YouTube. Check out the membership for more info.

Follow @CodeOpinion on Twitter

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.

Smarter Single Page Application with a REST API

How can you build a smarter Single Page Application with a REST API? The concepts have been since the beginning of the web, yet have somehow lost their way in modern REST API that drives a Single Page Application or Mobile Applications. Here’s how to guide clients based on state by moving more information from design time to runtime.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything that is in this post.

State

If you’re developing more than a CRUD application, you’re likely going to be driven by the state of the system. Apps that have Task Based UIs (hint: go read my post on Decomposing CRUD to Task Based UIs) are guiding users down a path of actions they can perform based on the state of the system.

The example throughout this post is the concept of a Product in a warehouse. If we have a tasks that let’s someone mark a Product as no longer being available for sale or it being available for sale, these tasks can be driven by the state of the Product.

If the given UI task is “Mark as Available” then the Product must be currently unavailable and we have a quantity on hand that’s greater than zero.

History of Clients

Taking a step back a bit, web apps were developed initially with just plain HTML (over 20 years ago for me). In its most basic form, a static HTML page contained a <form> that the browser rendered for the user to fill out and submit. The form’s action would point to a URI usually to a script, often written in Perl, in the cgi-bin folder on the webserver. The script would take the form data (sent via POST from the browser) and insert it into a database, send an email, or whatever the required behavior was.

As web apps progressed, instead of the HTML being in a static file, it was dynamically created by the server. But it was still just plain HTML.

The browser was the client. HTML was the content it’s consuming.

Modern Clients

As web apps progressed with AJAX (XMLHttpRequest) instead of using HTML forms, Javascript was used to send the HTTP request. The browsers turned more into the Host of the application which was written in Javascript.

Now, Javascript is the client. JSON is the content it’s consuming.

Runtime vs Design Time

Single Page Application with a REST API

When the Browser is the client consuming HTML, it understands how to render HTML. HTML has a specification. The browser understands how to handle a <form> tag or a <button>. It was driven by the HTML at runtime.

In modern SPAs consuming JSON, the data itself is unstructured. Each client has to be created uniquely based on the content it receives. This has to be developed at design time when creating the javascript client.

This is a shift from runtime to design time in modern clients.

When developing a SPA, you may leverage something like OpenAPI to generate code to use in the SPA/clients to make the HTTP calls to the server. But you must understand as a developer, at design time (when developing) when to make a call to the server.

To use my earlier example of making a product available for sale, if you were developing a server-side rendered HTML web app, you wouldn’t return the form apart of the HTML if the product couldn’t be made available. You would do this because on the server you have the state of the product (fetched from the database). If you’re creating a SPA, you’re likely putting that same logic in your client so you can conditionally show UI elements. It wouldn’t be a great experience for the user to be able to perform an action, then see an error message because the server/api threw a 400 because the product is not in a state to allow it to be available.

Hypermedia

Hypermedia is what is used in HTML to tell the Browser what it can do. As I mentioned earlier, a <form> is a hypermedia control.

When a user clicks the submit button, the browser is going to send a POST request to /mvc/save?Sku=123

The HTML is telling the Client (Browser) that his action is available.

Another hypermedia control is the <a> tag that is when clicked by the user in the browser, will then making an HTTP call to /mvc/images?Sku=abc123 and then render the returned HTML.

The HTML is telling the Client (Browser) where else it can navigate to.

The server is serving HTML that is providing the client with this information at runtime.

HTTP APIs

The vast majority of modern HTTP APIs serving JSON, do not provide any information in the content (JSON) about what actions or other resources the consuming client (SPA) can take. Meaning, we provide no information at runtime. All of that has to be figured out at design time.

In the example of a product, your typical response might look like this:

But how does your SPA know if it can mark the product as available for sale? Again, this is based on the state of the product, which your server knows since its returning you the state.

We can take the same concepts as forms and links and apply them to our modern HTTP APIs.

Now we’ve provided information at runtime to the client. If our client is developed at design time to know these links/actions may or may not exist, it can develop the appropriate UI to either show or hide certain functionality.

Now instead of being 100% design time, we’ve shifted more towards runtime.

You will still need to know (via OpenAPI) at design time, all the information about the routes you will be calling, and their results, however, you can now have the server return JSON that can guide the client based on state.

Media Types

This isn’t a new idea at all. Hypermedia in HTTP APIs exists with defined media types. Check out:

HAL http://stateless.co/hal_specification…
JSON API https://jsonapi.org/
Siren https://github.com/kevinswiber/siren
Hydra https://www.hydra-cg.com/spec/latest/…
JSON-LD https://json-ld.org/

Source Code

Developer-level members of my CodeOpinion YouTube channel get access to the full source for the working demo application available in a git repo. Check out the membership for more info.

Additional Related Posts

Follow @CodeOpinion on Twitter

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.

Is a REST API with CQRS Possible?

If you’re developing an HTTP API (what most would call REST) how does that fit alongside a Task-Based UI and CQRS? How can you create a REST API with CQRS? For starters, resources don’t need to map to Entities. Second, HTTP Methods don’t need to map to CRUD. Resources can be whatever you want them to be. In which case, they can be commands and queries.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything that is in this post.

Resources

In some of my recent videos/blogs about Task Based UIs, I’ve had people ask how CQRS can work with “REST”. The issue is what most think is REST, isn’t actually REST at all. There’s this notion that resources need to revolve around Entities. Specifically that HTTP methods map to CRUD (Create, Read, Update, Delete) around entities.

This is incorrect.

Here’s an example of a typical example revolving around Entities and CRUD. In this case, my entity is a Product in a warehouse.

Then the common pattern is to use HTTP methods mapping to CRUD. Although HTTP methods don’t fully map to CRUD, this again is the common pattern.

So for our Product Entity, our typical API would have an API following this CRUD pattern.

Resources

This misconception that resources should be entities (such as a Product) causes confusion on how CQRS can be applied. The reality is that a resource is whatever you want it to be. It can be an Entity, but it can also represent a Command or a Query.

From the Mozilla Developer Docs:

The target of an HTTP request is called a “resource”, whose nature isn’t defined further; it can be a document, a photo, or anything else. Each resource is identified by a Uniform Resource Identifier (URI) used throughout HTTP for identifying resources.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Identifying_resources_on_the_Web

An Resource can be anything. It does not need to be an entity. It can represent a command or a query.

REST API with CQRS

In order to really apply CQRS, you’re better served with a Task Based UI that will guide users to invoke commands. Command and Queries will be resources.

REST API with CQRS

Making your commands and queries resources makes them explicit. If you were to have only resources as entities, then you would have to infer which command to call for each POST/PUT/PATCH/DELETE based on the request body.

UI Composition

The next most common question is how to do UI composition. If there are multiple services that have resources that need to be called in order to get the relevant data for a UI, how is that done?

One option is to do the composition on the client. The client will call each service to get the relevant data it needs.

This means that there will be a call to each service. For the SKU, name, Description, since that’s owned by the catalog, there will be a call to that resource.

REST API with CQRS

And the same for all other data required, such as the price from Sales.

REST API with CQRS

The other option is to do the composition on the server. This is often referred to as Backend for Frontend.

REST API with CQRS

The server will call the relevant services and compose all the data into one resource that will be returned to the client with all the relevant data it needs.

Source Code

Developer-level members of my CodeOpinion YouTube channel get access to the full source for the working demo application available in a git repo. Check out the membership for more info.

Additional Related Posts

Follow @CodeOpinion on Twitter

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.