Documentation / Guides / Integrations / Cloud / AWS EKS
AWS EKS logo

Secrets Management for AWS EKS

This guide will show you how to provision an application running on EKS with the secrets it needs. To make life easy, you can use the demo app from the Getting Started guide or deploy your own custom app and follow along.

You will be able to grant your app access to the required secrets just by having your pods assume an IAM role, using the AWS Identity Provider.

Before you begin

Before you start using SecretHub with AWS EKS, make sure you have completed the following steps:

  1. Set up SecretHub on your workstation.
  2. Configure your AWS credentials.
  3. Install kubectl.
  4. Install eksctl, the official CLI for EKS.
  1. Set up SecretHub on your workstation.
  2. Configure your AWS credentials.
  3. Install kubectl.
  4. Configure the AWS and SecretHub providers.
  5. Declare the following variables:
variable "vpc_id" {
  description = "The VPC the EKS cluster lives in."
}

variable "subnets" {
  type        = list(string)
  description = "The subnets the EKS cluster lives in. A minimum of 2 AZs is required."
}

Step 1: Create Cluster

First, create a Kubernetes cluster:

eksctl create cluster --name secrethub-demo-cluster

data "aws_eks_cluster" "cluster" {
  name = module.cluster.cluster_id
}

data "aws_eks_cluster_auth" "cluster" {
  name = module.cluster.cluster_id
}

provider "kubernetes" {
  host                   = data.aws_eks_cluster.cluster.endpoint
  cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data)
  token                  = data.aws_eks_cluster_auth.cluster.token
  load_config_file       = false
  version                = "~> 1.9"
}

module "cluster" {
  source          = "terraform-aws-modules/eks/aws"
  cluster_name    = "secrethub-demo-cluster"
  cluster_version = "1.17"
  subnets         = var.subnets
  vpc_id          = var.vpc_id

  worker_groups = [
    {
      instance_type = "t2.medium"
      asg_max_size  = 1
    }
  ]
}

Step 2: Enable IAM Roles for Service Accounts

Next, to allow your pods to assume IAM roles and have secrets provisioned to them, enable IAM Roles for Kubernetes Service Accounts (IRSA):

eksctl utils associate-iam-oidc-provider --cluster secrethub-demo-cluster --approve

locals {
  oidc_issuer = data.aws_eks_cluster.cluster.identity.0.oidc.0.issuer
}

resource "aws_iam_openid_connect_provider" "cluster" {
  client_id_list  = ["sts.amazonaws.com"]
  thumbprint_list = [data.tls_certificate.cluster.certificates.0.sha1_fingerprint]
  url             = local.oidc_issuer
}

data "tls_certificate" "cluster" {
  url = local.oidc_issuer
}

Step 3: Create IAM Role & Service Account

The first thing to set up for your app is an IAM role, along with a Kubernetes Service Account that’s coupled to it.

Use eksctl to create a new Kubernetes service account named demo-app, along with a corresponding IAM Role:

eksctl create iamserviceaccount \
  --name demo-app \
  --cluster secrethub-demo-cluster \
  --attach-policy-arn <your-policy-arn> \
  --approve

eksctl requires at least 1 IAM policy to be attached to the role it creates, so if you don’t have an IAM policy for your app, go to the Create Policy page on the AWS Console and create one, e.g.:

{
    "Version": "2012-10-17",
    "Statement": {
        "Effect": "Allow",
        "Action": [
            "sts:GetCallerIdentity"
        ],
        "Resource": "*"
    }
}

To get the name of the created IAM Role, run:

kubectl get sa demo-app -o yaml

The role’s ARN is the the value of the eks.amazonaws.com/role-arn key. Take note of it, as you will need it in the next step.

resource "aws_iam_role" "secrethub_demo" {
  name               = "SecretHubDemoEKSRole"
  description        = "Role for SecretHub demo app"
  assume_role_policy = data.aws_iam_policy_document.demo_app_assume_role.json
}

resource "kubernetes_service_account" "demo_app" {
  automount_service_account_token = true
  metadata {
    name = "demo-app"

    annotations = {
      "eks.amazonaws.com/role-arn" = aws_iam_role.demo_app.arn
    }
  }
}

data "aws_iam_policy_document" "demo_app_assume_role" {
  statement {
    actions = ["sts:AssumeRoleWithWebIdentity"]

    principals {
      type        = "Federated"
      identifiers = ["${aws_iam_openid_connect_provider.cluster.arn}"]
    }

    condition {
      test     = "StringEquals"
      variable = "${replace(local.oidc_issuer, "https://", "")}:sub"
      values   = ["system:serviceaccount:default:demo-app"]
    }
  }
}

Step 4: Create a KMS key

Next, to allow your secrets to be encrypted and decrypted, set up a KMS key.

  1. Go to the Create Customer Managed Key page on the AWS Console.
  2. Enter an alias (e.g. MyServiceKey) and optionally a description for the key and click Next.
  3. Add any tags you like and click Next.
  4. Select any users or roles you would like as Key Administrators and click Next. Make sure your own IAM user or a role you have access is selected or that you select it on the next page as a Key User.
  5. Select the role created in the previous step as a Key User, set any other preferred Key Users and then click Next. A role or user you have access to should either be a Key User or a Key Administrator.
  6. Create the KMS key by clicking Finish.
  7. Take note of the id of the newly created key (e.g. 1234abcd-12ab-34cd-56ef-1234567890ab), you’ll need it in the next step.
resource "aws_kms_key" "secrethub_auth" {
  description = "KMS key to facilitate SecretHub authentication"
}

data "aws_iam_policy_document" "secrethub_auth" {
  statement {
    actions   = ["kms:Decrypt"]
    resources = [aws_kms_key.secrethub_auth.arn]
    effect    = "Allow"
  }
}

resource "aws_iam_policy" "secrethub_auth" {
  name        = "SecretHubAuth"
  description = "Allow SecretHub authentication using KMS"
  policy      = data.aws_iam_policy_document.secrethub_auth.json
}

resource "aws_iam_role_policy_attachment" "demo_app_secrethub_auth" {
  role       = aws_iam_role.demo_app.name
  policy_arn = aws_iam_policy.secrethub_auth.arn
}

Step 5: Setup SecretHub Service Account

With the IAM role and KMS key in place, we can go ahead and create a SecretHub service account for the app.

Run the following command and you’ll be prompted for the name of the role, the id or ARN of the KMS key and the region the KMS key is in:

secrethub service aws init your-username/demo --permission read

Setting --permission read automatically creates an access rule to give the newly created service account read access on the demo repo.

variable "secrethub_username" {
  description = "Your SecretHub username"
}

resource "secrethub_service_aws" "demo_app" {
  repo        = "${var.secrethub_username}/demo"
  role        = aws_iam_role.secrethub_demo.name
  kms_key_arn = aws_kms_key.secrethub_auth.arn
}
resource "secrethub_access_rule" "demo_app" {
  account_name = secrethub_service_aws.demo_app.id
  dir          = "${var.secrethub_username}/demo"
  permission   = "read"
}

As you may have noticed, secrethub service aws init – in contrary to the generic secrethub service init command – does not output a credential.

That’s because applications on AWS do not need it anymore: as long they take on the specified role, they can automatically get their secrets from SecretHub.

Step 6: Create Kubernetes Deployment

Next, to configure your deployment you can write a demo-app.yml file similar to this one:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    run: app
  name: demo-app
spec:
  replicas: 1
  selector:
    matchLabels:
      run: app
  template:
    metadata:
      labels:
        run: app
    spec:
      serviceAccountName: demo-app
      containers:
      - image: secrethub/demo-app
        name: demo-app
        env:
        - name: DEMO_USERNAME
          value: "secrethub://your-username/demo/username"
        - name: DEMO_PASSWORD
          value: "secrethub://your-username/demo/password"
        - name: SECRETHUB_IDENTITY_PROVIDER
          value: "aws"
        ports:
        - containerPort: 8080

The line serviceAccountName: demo-app configures the pods to use the previously created Kubernetes service account, which has access to the required secrets.

This image runs the demo app, with the secrethub run command and exposes it on port 8080. You can follow our Docker guide to use a custom docker image for your app.

In the env section the environment variables DEMO_USERNAME and DEMO_PASSWORD (used by the demo app) are configured to reference the previously created secrets.

Finally, all you need to do is tell the app it’s running on AWS by setting the SECRETHUB_IDENTITY_PROVIDER environment variable to aws. This will ensure the IAM Role of the pod is used for authenticating to SecretHub.

Step 7: Use the mutating webhook

If you don’t want to include the SecretHub binary in your image, you can use the SecretHub Mutating Webhook to automatically mount the CLI from a volume and wrap the entrypoint.

To use it, you need to install the webhook on your cluster once, which can be done in multiple ways.

Once the webook is installed, add an annotation to the pod with the container that needs to be mutated: secrethub.io/mutate: demo-app. If there’s multiple containers that need to be mutated, pass them as a comma-separated list: secrethub.io/mutate: app1,app2.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    run: app
  name: demo-app
spec:
  replicas: 1
  selector:
    matchLabels:
      run: app
  template:
    metadata:
      labels:
        run: app
      annotations:
        secrethub.io/mutate: demo-app
    spec:
      serviceAccountName: demo-app
      containers:
      - name: demo-app
        image: secrethub/demo-app
        command: ["demo", "serve"]
        args: ["--host", "0.0.0.0", "--port", "8080"]
        env:
        - name: DEMO_USERNAME
          value: secrethub://your-username/demo/username
        - name: DEMO_PASSWORD
          value: secrethub://your-username/demo/password
        - name: SECRETHUB_IDENTITY_PROVIDER
          value: aws
        ports:
        - containerPort: 8080

Step 8: Deploy app

Finally, to deploy the SecretHub demo app, run:

kubectl apply -f demo-app.yml

Step 8: Verify that it works

Now let’s see the app running!

Run the following command to forward data from your local 8080 port to one of the pods in the deployment:

kubectl port-forward deployment/demo-app 8080

Then visit: http://127.0.0.1:8080 and you should see the app running:

Screenshot of the demo app running
Demo app running

See also

SecretHub logo on dark background without text
We build software to make development, operations, and security into what it should be - easy and accessible
© 2018- SecretHub. All rights reserved. Privacy Policy Terms & Conditions