Sponsor: Do you build complex software systems? See how NServiceBus makes it easier to design, build, and manage software systems that use message queues to achieve loose coupling. Get started for free.
How do you manage your App Configuration? Things like connection strings, hostnames, 3rd party API keys, and application-specific settings like log levels. You can use config files, environment variables, and external configuration services. Let me sort it out and explain the different situations so you can choose the best option. It’s not a one size fits all.
YouTube
Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.
Included
It’s typical to include your configuration alongside your application. This could be in code, compiled into your binary, or a separate configuration file that’s read on the startup or runtime of your app. You just have to include the configuration in your deployment pipeline.
Three issues with doing this may be applicable in your context. The first is having multiple environments. Your configuration needs to change or include different configurations at deployment time. For example, you would likely have a different configuration for a staging environment than production because you’ll have different connection strings, etc.
Also, with connection strings, sensitive information such as usernames and passwords often is required. Depending on your infrastructure or environment, you might be connecting to something that requires API keys, username/password, etc. In these cases, you might not want any sensitive information in code and, ultimately, source control or configuration files in plain text. Lastly, because the configuration is static, the app often needs to be restarted if you need to change the configuration because it usually loads the configuration on startup. You lose the ability to change configuration while the application is running dynamically. A simple example is wanting to change the log level to verbose while the app is running for a short period so you can track down an issue in production.
Environment
If you have multiple environments and configurations, you need to tell your application at startup what environment it’s in so it can load the correct configuration.
One solution to this is using environment variables. Here’s an example of a docker compose file to specify an environment variable that we can define, and then in our application, look for this environment variable to determine which configuration to load.
There are different ways of doing this, depending on your tooling. Here’s the same example if you were using an AWS ECS task.
External Store
If you have sensitive data, such as API Keys, Username/Password, etc, you might want to use an external store that supports encrypting any of these type of secrets. The way it works is when your application starts up, instead of loading a static configuration file or compiled code, you reach out to the configuration store to get the values you need. For example, if you require a connection string to connect to your database.
After you’ve got the value from the configuration store, you can keep it in the application’s memory and use it whenever you need to connect to the database.
Depending on your configuration store, it may also support a callback mechanism or a polling interval to fetch any new configuration values that have changed. This allows you to change the current configuration loaded in the app at runtime while the application is running without needing to be restarted.
But everything has trade-offs, and more moving parts can fail if you introduce a configuration store. What happens when your application starts and tries to load the configuration from the store but can’t access it, or the store is unavailable? You need a local cache alongside your application from the last previous known good configuration.
A local cache allows the application to start with the last working configuration without requiring the configuration store to be available. To create this cache, anytime you can successfully fetch the configuration from the store, you persist that alongside your application in its hosted environment.
If you were in a scaled-out environment, you could also share this cache with other instances.
The cache is ultimately a fallback incase of available issues with the configuration store.
Context
As mentioned, it depends on your system what type of app configuration is best. There’s nothing wrong with using static configuration files if you aren’t concerned about sensitive data (secrets) or need dynamic configuration at runtime. You can define an environment variable that you would use in code at startup of your application to determine which configuration to load.
If you have sensitive data, you might be better served storing those in a configuration store. It doesn’t have to be the entire configuration, just those secrets.
Also, if you have certain configuration values that need to be dynamic, those could be in a configuration store that supports callbacks if the value changes so your application can update while it’s running, not requiring a restart.
But as always, there are trade-offs. Configuration stores add complexity. Somethings to consider are:
- If you cannot reach them at startup, can your application load? You want a fallback/cache.
- Does the store support pushing changes, or must you poll?
- How easy is it to debug the values coming from the configuration? It can sometimes be pretty difficult to get good visibility in production environments.
- Caution: if you’re making changes at runtime
Join CodeOpinon!
Developer-level members of my Patreon or YouTube channel get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.