Build Keyless Apps by Extending Native AWS Identity
If you’ve tried to implement secrets management into your infrastructure, then you’ve likely run into this problem:
How can you build your app so it can independently load the passwords and API keys it needs, without placing some magic keyfile somewhere?
We’ll show you how we eliminated this last secret on AWS and how you can do the same for your infrastructure.
This is Part 2 of a series to help you automatically load secrets into your app the moment it actually needs it, instead of having to place secrets in source code, CI environment variables or config files.
The Bootstrap Problem
The goal of secrets management is to get your app to automatically provision itself with the secrets it needs, without you having to hardcode them everywhere. It gives you things like encryption, access control, audit logging, and easy rotation.
But in a world without hardcoded secrets, how do your apps authenticate to the secrets manager?
The app would need to be pre-provisioned with some sort of credential, before being able to provision itself with the actual application secrets.
This introduces yet another secret you need to manage, another turtle.
Sure, having to provision an app with only a single secret is a lot better than having to do so with dozens of secrets more, but it does add extra operational complexity.
If you want your infrastructure to easily scale up and down, you can’t be manually pre-provisioning new instances with a key all the time. That just doesn’t scale.
The problem gets worse the more immutable and elastic your infrastructure becomes. When deployments are autonomous, new instances should be able to go from 0 to 100 all by themselves, not from 1 to 100.
How We Solved It on AWS
So, what if we could eliminate that pre-provisioning step entirely?
The ideal situation would be for your app to be able to get to its secrets just by virtue of the app being the app. That way, the app could be made keyless.
The key to keyless is identity. AWS already has a rather powerful identity system that everyone knows, trusts, and uses: IAM. When combined with STS, your apps can actually take on an IAM role on the fly, and get certain privileges without having to carry a static credential.
It turns out that this ‘STS dance’ is not just limited to AWS services talking to other AWS services (e.g. an ECS container getting a file from an S3 bucket). It’s simply an HTTP endpoint that non-AWS software can also use.
So what we’ve done is this: client-side, we’re creating and signing a
GetCallerIdentity request (which is merely a “Who Am I” within AWS), but instead of sending it to the AWS API directly, we’re proxying it to our servers first.
If our servers get confirmation from AWS that your app really is your app, our system will trust it too. We’re taking the identity the app has within AWS and extending it outside of the AWS ecosystem.
With a solution in place for the authentication part, we’re not entirely there yet: SecretHub is designed to be end-to-end encrypted, so it can be offered as a service where you don’t have to host anything.
But that does mean that authenticating is not enough to get to your secrets: They still need to be decrypted client-side.
And that’s where KMS comes in, which is AWS’s native ‘encryption-as-a-service’ solution that lets you encrypt and decrypt data without you ever seeing the keys behind it.
We’ve integrated KMS into the SecretHub client, so that during the bootstrap phase of your app, the key material automatically gets decrypted using a KMS key you’ve specified.
That was the last piece in enabling your app to automatically fetch and decrypt its secrets without needing human intervention. So now, new instances can go from all the way from 0 to 100 by themselves!
How to Make Your App Keyless
It may seem like this adds complexity, but you won’t notice it because under the hood everything is taken care of automatically.
After doing this once for your app, all you really need to do is tell the app it’s running on AWS:
secrethub run --identity-provider aws -- node app.js
That’s it: end-to-end encrypted secrets management without the hassle of keyfiles. 🙌
And it works everywhere on AWS, check out the guides on how to do it on your preferred service:
⛅ Coming to a Cloud Near You
If AWS isn’t your (only) cloud, don’t worry: we’re already working on other cloud providers as well. The same principles can be applied elsewhere:
As long as there is a cloud-native way for services to prove their identity and an encryption/decryption service, your apps can be made keyless over there.
To be notified when keyless secrets management is available for the cloud you’re on, follow us on Twitter!