AWS ECS logo

Secrets Management for AWS ECS

Running tasks in the isolated environment of a container can make your life a lot easier. However, letting your containerized application get access to secrets is not straightforward. Especially when running these containers in AWS Elastic Container Service.

Using the SecretHub AWS Identity Provider can help you out. This guide will show you how to use the AWS Identity Provider to access secrets in an ECS Task using SecretHub.

To make life easy, you can use demo app from the Getting Started guide to have something to deploy to ECS.

Overview of using SecretHub in an ECS Task

Before you begin

Before you start, make sure you have completed the following steps:

  1. Set up SecretHub on your workstation.
  2. Configure your AWS credentials.
  3. If you are using Terraform, installed the SecretHub Terraform Provider.

Step 1: Create an IAM Role for ECS

The first thing we need for the AWS Integration to work, is an IAM role.

  1. Go to the Create role page on the AWS Console.
  2. Select AWS Service and Elastic Container Service as trusted entity.
  3. Select Elastic Container Service Task as use case and continue by clicking Next: Permissions.

First step in creating IAM Role

  1. Select any Policies your ECS Task needs and then click Next: Tags. For using SecretHub no specific policy is needed.
  2. Add any tags you like and click Next: Review.
  3. Set a descriptive Role name (e.g. SecretHubDemoECSTaskRole) and description and click Create role.

Run the following command to create an IAM role with the required policy attached:

export ROLE_ARN=$(
  aws iam create-role \
    --role-name SecretHubDemoECSTaskRole \
    --description "Role for SecretHub demo app" \
    --assume-role-policy-document '{
      "Version": "2012-10-17",
      "Statement": [
      {
        "Effect": "Allow",
        "Action": [
          "sts:AssumeRole"
        ],
        "Principal": {
          "Service": ["ecs-tasks.amazonaws.com"]
        }
      }
      ]
    }' \
    --query "Role.Arn" \
    --output text
)
resource "aws_iam_role" "secrethub_demo" {
  name               = "SecretHubDemoECSTaskRole"
  assume_role_policy = data.aws_iam_policy_document.ecs_assume_role.json
  description        = "Role for SecretHub demo app"
}

data "aws_iam_policy_document" "ecs_assume_role" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service"
      identifiers = ["ecs-tasks.amazonaws.com"]
    }
  }
}

Step 2: Create a KMS key

Next, we have to setup a KMS Key to use for encryption and decryption.

  1. Go to the Create Customer Managed Key page on the AWS Console.
  2. Enter an alias (e.g. SecretHub-Demo-Service-Key) and optionally a description for the key and click Next.
  3. Add any tags you like and click Next.
  4. Select any users or roles you would like as Key administrators and click Next. Make sure your own IAM user or a role you have access is selected or that you select it on the next page as a Key user.
  5. Select the role you previously created as a Key user and set any other preferred key users and then click Next. A role or user you have access to should either be a Key user or a Key Administrator.

Select the correct Key user for the KMS key

  1. Create the KMS key by clicking Finish.
  2. Take note of the id of the newly created key (e.g. 1234abcd-12ab-34cd-56ef-1234567890ab), you’ll need it in the next step.

To create a KMS key, run:

export KMS_KEY_ARN=$(
  aws kms create-key \
  --description "KMS key to facilitate SecretHub authentication" \
  --query "KeyMetadata.Arn" \
  --output text
)

Then, to allow the IAM role to use this KMS key, create a policy by running:

export POLICY_ARN=$(
  aws iam create-policy \
    --policy-name SecretHubAuth \
    --description "Allow SecretHub authentication using KMS" \
    --policy-document '{
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "kms:Decrypt"
          ],
          "Resource": ["'$KMS_KEY_ARN'"]
        }
      ]
    }' \
    --query "Policy.Arn" \
    --output text
)

Attach this policy to the previously create IAM role, by running:

aws iam attach-role-policy \
  --role-name SecretHubDemoECSTaskRole \
  --policy-arn $POLICY_ARN
resource "aws_kms_key" "secrethub_auth" {
  description = "KMS key to facilitate SecretHub authentication"
}

data "aws_iam_policy_document" "secrethub_auth" {
  statement {
    actions   = ["kms:Decrypt"]
    resources = [aws_kms_key.secrethub_auth.arn]
    effect    = "Allow"
  }
}

resource "aws_iam_policy" "secrethub_auth" {
  name        = "SecretHubAuth"
  description = "Allow SecretHub authentication using KMS"
  policy      = data.aws_iam_policy_document.secrethub_auth.json
}

resource "aws_iam_role_policy_attachment" "secrethub_demo_auth" {
  role       = aws_iam_role.secrethub_demo.name
  policy_arn = aws_iam_policy.secrethub_auth.arn
}

Step 3: Setup SecretHub Service Account

With the IAM role and KMS key in place, we can go ahead and create a SecretHub service account for the app.

Run the following command and you’ll be prompted for the name of the role, the id or ARN of the KMS key and the region the KMS key is in:

secrethub service aws init your-username/demo --permission read

Setting --permission read automatically creates an access rule to give the newly created service account read access on the demo repo.

variable "secrethub_username" {
  description = "Your SecretHub username"
}

resource "secrethub_service_aws" "demo_app" {
  repo        = "${var.secrethub_username}/demo"
  role        = aws_iam_role.secrethub_demo.name
  kms_key_arn = aws_kms_key.secrethub_auth.arn
}
resource "secrethub_access_rule" "demo_app" {
  account_name = secrethub_service_aws.demo_app.id
  dir          = "${var.secrethub_username}/demo"
  permission   = "read"
}

As you may have noticed, secrethub service aws init – in contrary to the generic secrethub service init command – does not output a credential.

That’s because applications on AWS do not need it anymore: as long they take on the specified role, they can automatically get their secrets from SecretHub.


Step 4: Create ECS Cluster

Before you can run a task on ECS, you have to create an ECS cluster:

  1. Go to the Create Cluster wizard.
  2. Select Networking only and click Next step.
  3. Enter a name for the cluster (e.g. SecretHubDemoCluster) and click Create.

Create a cluster by running the following command:

aws ecs create-cluster --cluster-name SecretHubDemoCluster
resource "aws_ecs_cluster" "secrethub_demo" {
  name = "SecretHubDemoCluster"
}

Step 5: Create Task Definition

The next step is to create a task definition that describes the task you’re going to run.

To run the task you’ll need the application in a Docker image. You can use our secrethub/demo-app image or create and publish your own.

  1. Go to the Create new Task Definition wizard.
  2. Select FARGATE and click Next step.
  3. Enter a Task Definition Name (e.g. SecretHubDemo).
  4. Select the previously created IAM role as Task Role.
  5. Set Task memory to 0.5GB and Task CPU to 0.25 vCPU.
  6. Click Add container.
  7. Enter app as Container name.
  8. Enter the location of the demo app image in the Image field: secrethub/demo-app.
  9. Scroll down to ENVIRONMENT section and add an Environment variable with the key SECRETHUB_IDENTITY_PROVIDER and aws as value.
  10. Provision the demo app with secrets by populating environment variables DEMO_USERNAME and DEMO_PASSWORD with secret references: Environment variables for ECS Task Definition
  1. Click Add.
  2. Click Create.

First, get the ARN of the ecsTaskExecutionRole by running:

export EXECUTION_ROLE_ARN=$(
  aws iam get-role \
  --role-name ecsTaskExecutionRole \
  --query "Role.Arn" \
  --output text
)

Run the following command to register the task definition:

export TASK_DEFINITION_ARN=$(
  aws ecs register-task-definition --cli-input-json '{
      "family": "SecretHubDemo",
      "cpu": "256",
      "memory": "512",
      "requiresCompatibilities": ["FARGATE"],
      "networkMode": "awsvpc",
      "executionRoleArn": "'$EXECUTION_ROLE_ARN'",
      "taskRoleArn": "'$ROLE_ARN'",
      "containerDefinitions": [
        {
          "name": "app",
          "image": "secrethub/demo-app",
          "portMappings": [
            {
              "containerPort": 8080,
              "hostPort": 8080
            }
          ],
          "environment": [
            {
              "name": "SECRETHUB_IDENTITY_PROVIDER",
              "value": "aws"
            },
            {
              "name": "DEMO_USERNAME",
              "value": "secrethub://your-username/demo/username"
            },
            {
              "name": "DEMO_PASSWORD",
              "value": "secrethub://your-username/demo/password"
            }
          ]
        }
      ]
    }' \
    --query "taskDefinition.taskDefinitionArn" \
    --output text
)

As you can see, the task definition is completely free of secrets.

SECRETHUB_IDENTITY_PROVIDER is set to aws so that IAM is used to authenticate as the previously created AWS service account.

data "aws_iam_role" "ecs_execution_role" {
  name = "ecsTaskExecutionRole"
}

resource "aws_ecs_task_definition" "secrethub_demo" {
  family                   = "SecretHubDemo"
  cpu                      = "256"
  memory                   = "512"
  requires_compatibilities = ["FARGATE"]
  network_mode             = "awsvpc"
  execution_role_arn       = data.aws_iam_role.ecs_execution_role.arn
  task_role_arn            = aws_iam_role.secrethub_demo.arn
  container_definitions    = jsonencode(local.container_definitions)
}

locals {
  container_definitions = [
    {
      name  = "app"
      image = "secrethub/demo-app"
      portMappings = [
        {
          containerPort = 8080
          hostPort      = 8080
        },
      ]
      environment = [
        {
          name  = "SECRETHUB_IDENTITY_PROVIDER"
          value = "aws"
        },
        {
          name  = "DEMO_USERNAME"
          value = "secrethub://your-username/demo/username"
        },
        {
          name  = "DEMO_PASSWORD"
          value = "secrethub://your-username/demo/password"
        },
      ]
    },
  ]
}

As you can see, the task definition is completely free of secrets.

SECRETHUB_IDENTITY_PROVIDER is set to aws so that IAM is used to authenticate as the previously created AWS service account.


Step 6: Launch Task

The only thing left to do is launch the task in the ECS Cluster:

  1. Go to the ECS Cluster Overview and select the cluster you just created.
  2. Select the Tasks tab and click Run new Task.
  3. Set Launch type to FARGATE.
  4. Select the Task Definition you just created.
  5. Select the desired Cluster VPC and Subnets (default will suffice for testing purposes).
  6. Under Security Groups make sure that there is a rule that allows TCP traffic from your IP (or anywhere) on port 8080 and click Save.
  7. Set Auto-assign public IP to ENABLED.
  8. Click Run Task.
  9. Click on the Task id in the list of tasks.
  10. When the Status of the container is RUNNING, look for the public IP can be found in the Network section of the Task.

To launch the ECS cluster, you first need to create a security group for it.

Pick a VPC with one or more public subnets and run the following:

export VPC_ID=<ID of your VPC>
export SUBNET_ID=<ID of a subnet in the VPC>

Then you can create a security group for the ECS service:

export SECURITY_GROUP_ID=$(
  aws ec2 create-security-group \
    --group-name SecretHubDemoSecurityGroup \
    --description "Allow public access to SecretHub demo app" \
    --vpc-id $VPC_ID \
    --query "GroupId" \
    --output text
)

To allow connections on port 8080, run:

aws ec2 authorize-security-group-ingress \
  --group-name SecretHubDemoSecurityGroup \
  --protocol tcp \
  --port 8080 \
  --cidr 0.0.0.0/0

Finally, create and deploy the service by running:

aws ecs create-service \
  --service-name SecretHubDemo \
  --cluster SecretHubDemoCluster \
  --launch-type FARGATE \
  --task-definition $TASK_DEFINITION_ARN \
  --network-configuration '
  {
    "awsvpcConfiguration": {
      "subnets": ["'$SUBNET_ID'"],
      "securityGroups": ["'$SECURITY_GROUP_ID'"],
      "assignPublicIp": "ENABLED"
    }
  }' \
  --desired-count 1

To get the public IP of the service go to the ECS page of the AWS console, click on your cluster’s name and switch to the Tasks tab. When the newly created task has the RUNNING status, click on its name. You can find the public IP of the task under the Network section.

variable "vpc_id" {}

variable "subnets" {
  type = list(string)
}

resource "aws_ecs_service" "secrethub_demo" {
  name            = "SecretHubDemo"
  cluster         = aws_ecs_cluster.secrethub_demo.id
  launch_type     = "FARGATE"
  task_definition = aws_ecs_task_definition.secrethub_demo.arn

  network_configuration {
    subnets          = var.subnets
    security_groups  = [aws_security_group.secrethub_demo.id]
    assign_public_ip = true
  }

  desired_count = 1
}

resource "aws_security_group" "secrethub_demo" {
  description = "Allow public access to SecretHub demo app"
  vpc_id      = var.vpc_id

  ingress {
    from_port   = 8080
    to_port     = 8080
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

Pick a VPC with one or more public subnets and configure the vpc_id and subnets variables. All that’s left now is to run terraform apply.

Note that the aws_ecs_service resource doesn’t offer the assigned public IP as an output, so you’ll have to dig that up in the AWS console or using the AWS CLI to see the application in action.

Visit http://<PUBLIC-IP>/ in your browser, and if all went well you’ve got an application that automatically provisioned itself with end-to-end encrypted secrets, yet doesn’t need to be provisioned with a key!


See also