Troubleshooting StackOverflow Exceptions

Is your .NET application randomly crashing? In Production? Without any relevant application logs?  You may be hitting a StackOverflowException if you see exception code 0xc00000fd in the event log. Here’s how to start troubleshooting StackOverflow Exceptions.

0xc00000fd

First take a look in your event log.  If you see a log similar to this:

Faulting application name: MyApp.exe, version: 1.0.0.0, time stamp: 0xdfd8c80b
Faulting module name: clr.dll, version: 4.7.2558.0, time stamp: 0x59d413ce
Exception code: 0xc00000fd

The important part here is the exception code 0xc00000fd is a StackOverflowException.

Troubleshooting a .NET StackOverflowException

First thing I would recommend is looking for any recursion in your codebase.  This is often likely the cause when you have unbounded recursion.  It may not be obvious if you have data structures that play a role in your application and specifically where recursion exists.

Debug Diagnostics Tool (DEBUGDIAG)

If you can’t determine where the StackOverflowException is occuring, first place to start is by downloading and installing the Debug Diagnostics Tool from Microsoft.

https://www.microsoft.com/en-us/download/details.aspx?id=49924

Once installed, run the “DebugDaig 2 Collection” application.  You will be immediately prompted with the following window:

If you’re running ASP.NET 4.x you will likely be selecting IIS, select the first option.  Otherwise, you likely are going to select a process or a NT Service (Windows Service).

On the follow window, click the Exceptions button.

DebugDiag

From here we can specify to capture a StackOverflow exception.  Change the Action Type to a Full userdump.

DebugDiag

Finally, click OK and Next to save the new Crash Rule.

Once your application hits a StackOverflowException and crashes, you will have a .dmp file in the rule folder.

DebugDiag

WinDbg

Once you have your memory dmp file, we are going to want to open it up with WinDbg.

You have a couple options for installing.  You can download Windows Driver Kit (WDK) which which require you to have latest Visual Studio with Desktop Development with C++ component installed.

https://go.microsoft.com/fwlink/?linkid=873060

Another option is to download the the new WinDbg Preview from the Windows Store.  This is a new version of WinDbg with a more modern look.

Once you have WinDbg, whatever version, you can now type in .loadby sos clr in the Command input.

WinDbg

Followed by !CLRStack as the next command, which should then provide you with your .NET call stack.

WinDbg

If you have any recommendations on WinDbg usage or other useful insights, please share them in the comments or on Twitter.

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.

ASP.NET Core Data Protection

ASP.NET Core Data ProtectionContinuing from my last post on Configuring ASP.NET Core behind a Load Balancer, the next hurdle you may run into is with ASP.NET Core Data Protection.

Specifically I was using Cookie Authentication (without Identity).

In this scenario, ASP.NET Core’s Data Protection must share the same key ring and app identifier for each instance of your application.  This means if you are load balanced across multiple containers or even machines, you must configure ASP.NET Core’s Data Protection system.

If you do not, the process that generates your authentication cookie (or bearer token) will be the only process that will be able to read it.

ASP.NET Core Data Protection

Thankfully, there are a a few different solutions that I’d like to point out across Azure, AWS and Redis.

Azure Blob Storage

There is an official package Microsoft.AspNetCore.DataProtection.AzureStorage that allows you to store your data protection keys in Azure storage.  Just use one of the overloads of the PersistKeysToAzureBlogStorage.

Redis

This is another official package Microsoft.AspNetCore.DataProtection.Redis that allows you to store it to Redis.  Use one the PersistToRedis methods to configure to your needs.

AWS S3 & KMS

If you are using AWS, you have a couple options as well with a couple community packages AspNetCore.DataProtection.Aws.S3 and AspNetCore.DataProtection.Aws.Kms.  You can find info and usage on GitHub.

Here’s how to quickly configure using using S3.

Key Storage Providers

Are you using any other key storage providers?  Please let me know in the comments or on Twitter.

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.

Configuring ASP.NET Core Behind a Load Balancer

ASP.NET Core behind a Load BalancerIf you deploy your ASP.NET Core web application to the cloud you are likely putting it behind a load balancer.  Here’s some quick info that might provide useful for how to Configure ASP.NET Core behind a load balancer.

Forwarded Headers

There are generally 3 headers which are added to the request header to tell your application about how it was forwarded from the load balancer.

  • X-Forwarded-For: List of comma space list of IP addresses of the original client and proxies that received the request.
  • X-Forwarded-Proto: The scheme from the original client and proxies.
  • X-Forwarded-Host: Original host header.

SSL Termination

If you are using a load balancer, it is common to have the load balance terminate the SSL connection and send the request to your application over HTTP.

Note: I strongly believe you should use SSL all the way through to your application.  HTTPS everywhere.

However, if you are using SSL termination, you can run into an issue of any links generated by the application will not be to HTTPS, but rather HTTP since that is how your application is running.

For example, if you are using Authentication middleware and the Authorize attribute, when a user is not logged in, and they are redirected to the LoginPath, they will be redirect to HTTP instead of the originating HTTPS, which they requested to begin with.

ASP.NET Core Behind a Load Balancer

What this does is updates the Request.Scheme with the X-Forwarded-Proto header so that all redirects link generation uses the correct scheme.

Note: By default, which seems confusing, is the UseForwardedHeaders without any parameters is to ForwardedHeaders.None.

HTTPS Everywhere

ASP.NET Core 2.1 has announced improvements for using HTTPS during development which should hopefully help push adoption for HTTP everywhere.

ASP.NET Core 2.1 makes it easy to both develop your app with HTTPS enabled and to configure HTTPS once your app is deployed. The ASP.NET Core 2.1 project templates have been updated to enable HTTPS by default. To enable HTTPS in production simply configure the correct server certificate.

If you are using SSL termination, I’d love to hear why and in what situations?  Please let me know in the comments or on Twitter.

Enjoy this post? Subscribe!

Subscribe to our weekly Newsletter and stay tuned.