How to inject secrets into Ansible playbooks with SecretHub

Ansible is an automation engine that helps developers to provision and configure systems and applications. Whether you configure and deploy to one machine or perform a complex rolling upgrade of an entire data center, every task and every policy is codified in one single source of truth: an Ansible playbook.

Playbooks often deploy secrets to machines or need secrets to access other resources: database passwords, API tokens, encryption keys. SecretHub allows you to remove all that secret data from playbooks and inject them at runtime. This enables you to freely collaborate on plaintext playbooks and instantly share knowledge throughout your team, without exposing any secrets.

Below we’ll explore how you can inject secrets into Ansible playbooks and how to deploy a service account using Ansible.


First some setup

To follow along, you will need a couple of things:

  • You have installed the SecretHub CLI and have successfully signed up for an account. If you haven’t yet done so, go do it now.
  • You have a working knowledge of Ansible and know your way around their documentation.
  • You have Ansible 2.5 installed.
  • You run a Unix system.

Running Windows?

You can still follow along the guide, but you’ll have to change a few commands to work on Windows. Pull requests are welcome on the examples repository so feel free to extend it. If you need help integrating Ansible with SecretHub on Windows, contact us we’ll help you get up and running.

To help you get started, we’ve prepared some example playbooks for you and put them on GitHub here.

Open up your terminal, clone this repository and cd into it:

git clone git@github.com:secrethub/ansible-examples.git
cd ansible-examples

To avoid having to rewrite all playbooks and type your SecretHub username over and over again, set an environment variable SECRETHUB_USERNAME to your SecretHub username. Ansible will use that variable later.

export SECRETHUB_USERNAME="<your_username>"

Now run the setup.sh script. This will create a SecretHub repository for you and pre-configure it with directories and secrets for the examples. The script is located under the 0-setup folder:

0-setup/setup.sh

Try running the tree command on the SecretHub repository:

secrethub tree ${SECRETHUB_USERNAME}/ansible-examples

You’ll see that the setup.sh script has created a directory for each step and already written a few secrets for your first playbook.

$ secrethub tree ${SECRETHUB_USERNAME}/ansible-examples
ansible-examples/
├── 1-read/
│   ├── db_password
│   └── db_user
├── 2-write/
└── 3-service/
    └── password

3 directories, 3 secrets

1. Injecting a secret into a playbook

Now that you have your repository setup, you can move on to actually reading a secret from SecretHub into a playbook.

To inject secrets into a playbook without seeing them yourselves, you can run local tasks that call the SecretHub CLI and register the output into variables that can be used later on.

For the purpose of this guide, you’ll use the secret variables in a local debug message in a local task that simply prints them out in debug messages. Naturally, you can use these variables in any way you’d like in your playbooks.

As you’ve seen before, the setup.sh script has created two secrets that we can read into our playbooks db_user and db_password.

To see the randomly generated db_password you can use the read command:

secrethub read ${SECRETHUB_USERNAME}/ansible-examples/1-read/db_password

Let’s walk through the playbook at 1-read/playbook.yml step by step.

---
- name: Load secrets from SecretHub
  hosts: localhost
  connection: local
  vars:
    # Construct the repository path from the environment.
    path:  "{{ lookup('env', 'SECRETHUB_USERNAME') }}/ansible-examples/1-read"
  tasks:
  - name: Load db_password secret into variable
    command: "secrethub read {{ path }}/db_password"
    register: db_password
  - name: Load db_user secret into a variable
    command: "secrethub read {{ path }}/db_user"
    register: db_user

- name: Deploy secrets to host
  hosts: localhost # We choose localhost here to avoid dependencies.
  tasks:
  - name: Print out the secrets (in real life you would use this for e.g. connecting to a database)
    debug: 
      msg:
        - "db_user={{ db_user.stdout }}"
        - "db_password={{ db_password.stdout }}"

The playbook executes the following steps:

  1. It retrieves your username from the environment using the lookup function to construct the SecretHub directory path.
  2. Then it executes a shell command that reads the db_password secret into a variable.
  3. Using the same mechanism, the db_user secret is registered into a variable.
  4. The second task is now able to inject the registered variables with the Jinja2 syntax {{ ... }} and use them in a debug statement. As the variables are actually the output of a shell command, you need the .stdout suffix to actually get the secret output.

Notice how the playbook.yml file does not contain any sensitive data? This is possible because all secrets are read from SecretHub and fed to the tasks at runtime. Also, no crytpo knowledge is required to understand this playbook. This means you can safely check the file into source control and share it with your team.

When you run the playbook, it prints a debug message with the db_user and db_password secrets it retrieves from SecretHub:

$ ansible-playbook 1-read/playbook.yml
[...]
TASK [Print out the secrets (in real life you would use this for e.g. connecting to a database)] ***
ok: [localhost] => {
    "msg": [
        "db_user=user1",
        "db_password=LtjPlhKncaS6NfcjdcpXkB"
    ]
}
[...]

Run the playbook for yourself and you’ll see a similar result. Note that the db_password will be different from the one shown above as it is randomly generated upon setup.

ansible-playbook 1-read/playbook.yml

Congratulations! You have just successfully injected secrets into a playbook.

Now let’s move on a more complex example where you will generate a secret on the fly and use it later on.


2. Generating a secret on the fly and injecting it in another task

Now that you know how to inject pre-existing secrets into a playbook, you can generate a secret from inside a playbook and inject that secret into a task.

Let’s walk through the playbook at 2-write/playbook.yml piece by piece:

---
- name: Load secrets from SecretHub
  hosts: localhost
  connection: local
  vars:
    path:  "{{ lookup('env', 'SECRETHUB_USERNAME') }}/ansible-examples/2-write"
  vars_prompt:
    - name: myvar
      prompt: "Please type in a value to write to 'myvar' in SecretHub"
      private: yes
  tasks:
  - set_fact: # This makes the prompted var accessible to the task definitions below
        myvar: "{{ myvar }}"
  - name: Write the prompted value to SecretHub
    shell: "echo \"{{ myvar }}\" | secrethub write {{ path }}/myvar"
  - name: Generate a random password
    command: "secrethub generate rand {{ path }}/password"
  - name: Read the generated password
    command: "secrethub read {{ path }}/password"
    register: password

- name: Use secret variables on the host
  hosts: localhost # We choose localhost here to avoid dependencies.
  tasks:
  - name: Print out the secrets (in real life you would use this for e.g. connecting to a database)
    debug: 
      msg:
        - "myvar={{ myvar }}"
        - "password={{ password.stdout }}"

Instead of reading secrets from SecretHub, this playbook prompts the user for a secret myvar and writes that to SecretHub. This variable is populated by a prompt for simplicity, but you can pipe any string into the write command.

Also, this playbook uses the generate command to generate a random password and write it to SecretHub. This password can then be used in different tasks.

The playbook performs the following steps:

  1. Ansible prompts you for the myvar variable. You can put any secret here.
  2. Then it executes a shell command that pipes myvar’s value to the write command, storing it in the SecretHub repository.
  3. Next, the generate command generates a random secret and automatically writes it to SecretHub. Using the read command, it registers the generated secret as a variable.
  4. The other task is now able to inject the registered variables with the Jinja2 syntax {{ ... }}.

Running the playbook produces the following results, printing a debug message with the myvar and password secrets:

$ ansible-playbook 2-write/playbook.yml
Please type in a value to write to 'myvar' in SecretHub:
[...]
TASK [Print out the secrets (in real life you would use this for e.g. connecting to a database)] ***
ok: [localhost] => {
    "msg": [
        "myvar=Hello Ansible!",
        "password=DzVaQb6ewseDPVi6nSVUny"
    ]
}
[...]

Run the playbook yourself and you’ll see a similar result. Note that the password will be different each time you run this playbook as it is randomly generated.

ansible-playbook 2-write/playbook.yml

Congratulations! You have just successfully created two secrets with a playbook and used them in a task.

Run the tree command again and you’ll see the playbook’s secrets are stored safely in your SecretHub repository.

secrethub tree ${SECRETHUB_USERNAME}/ansible-examples

Let’s move on to a different use case that uses a dedicated account to read a secret instead of your own personal account.


3. Deploying a service account with Ansible

Before, you’ve only written and read secrets from a local task. This means that all actions are performed using the SecretHub account you’ve configured locally. Sometimes, however, you want a service to have unattended access to secrets inside a SecretHub repository.

You can do this by creating a service account credential for the host and configure the CLI to use that account.

A service account is a non-human account that is tied to a repository. To read more about service accounts, see this page.

Now, let’s walk through the playbook 3-service/playbook.yml step by step.

---
- name: Load secrets from SecretHub
  hosts: localhost
  connection: local
  vars:
    # Construct the repository path from the environment.
    path: "{{ lookup('env', 'SECRETHUB_USERNAME') }}/ansible-examples/3-service"
  tasks:
  - set_fact: # This makes the path accessible to the task definitions below
        path: "{{ path }}"
  - name: Create a service account and give it read permission on the directory (may take a minute)
    shell: "secrethub service init {{ path }} --permission read"
    register: service_credential

- name: Deploy secrets to host
  hosts: localhost # We choose localhost here to avoid dependencies.
  vars:
    config_dir: "/tmp/ansible-examples/3-service/.secrethub"
  tasks:
  - name: Create a directory for this example
    file:
      path: "{{ config_dir }}"
      state: directory
      recurse: yes
  - name: "Write the service account credential to a file"
    copy: 
      dest: "{{ config_dir }}/credential"
      content: "{{ service_credential.stdout }}"
  - name: "Let the service account read the secret and write it to a file"
    environment:
      # Set the config dir envar so the CLI uses the credential stored in that directory instead.
      SECRETHUB_CONFIG_DIR: "{{ config_dir }}"
    shell: "secrethub read {{ path }}/password"
    register: password
  - name: Print out the password the service account has read
    debug: 
      msg:
        - "password={{ password.stdout }}"

Instead of using our own account to read the password secret from SecretHub, this playbook creates a service account and uses that account to read the password.

When you run the playbook, the following steps are executed:

  1. The playbook creates a service account using the service init command and gives it read permission on the directory with the secrets it needs. The service credential is registered as a variable service_credential.
  2. Another task creates a temporary config_dir directory to store the service account credential in.
  3. Then it writes the credential to the config directory.
  4. Next it reads the secret from SecretHub, but uses the credential stored in the directory the SECRETHUB_CONFIG_DIR environment variable points to.
  5. Finally, it prints out a debug statement with the password it has just read.

Running the playbook produces the following results, printing a debug message with the password secret read by the service account:

$ ansible-playbook 3-service/playbook.yml
[...]
TASK [Print out the password the service account has read] *********************
ok: [localhost] => {
    "msg": [
        "password=gDXh5zgAKT4zB91qTuyMq1"
    ]
}
[...]

Run the playbook yourself and you’ll see a similar result.

ansible-playbook 3-service/playbook.yml

Congratulations! You have now successfully deployed a service account to a host and allowed it to read a secret from SecretHub.

To verify that it was actually the service account that has read the secret, run the audit command:

secrethub audit ${SECRETHUB_USERNAME}/ansible-examples/3-service/password

Running the audit command should produce output a bit similar to this:

$ secrethub audit marc/ansible-examples/3-service/password
AUTHOR           EVENT                   IP ADDRESS       DATE
marc             create.secret           171.19.220.25    About a minute ago
marc             create.secret_version   171.19.220.25    About a minute ago
s-vzWIvJjGZrbM   read.secret_version     171.19.220.25    12 seconds ago
s-vzWIvJjGZrbM   read.secret             171.19.220.25    12 seconds ago

Note that the secret can be read by the service account outside of Ansible too, as long as you provide the SECRETHUB_CONFIG_DIR argument to point to the service credential.


Finishing up

A fun experiment that is left for the reader is to add some secrethub_path variables to your Ansible inventory and read a different secret for specific hosts.

Also, using a service account credential you can inject secrets into systems upon boot with the secrethub set command and store them on an in-memory filesystem using tmpfs. We’ll probably write a blog post on that one soon.

Once you’re done, you can remove the ansible-examples repository from your SecretHub account with this command:

secrethub repo rm ${SECRETHUB_USERNAME}/ansible-examples

And you can also remove the temporary config directory the third playbook created:

rm -r /tmp/ansible-examples

That’s it. You’re now able to use SecretHub to automatically inject secrets into Ansible playbooks. No need to hard-code them in playbooks and no more typing them in manually at runtime. All configuration is kept in one place without compromising security.

We’re working on an Ansible plugin to make injecting secrets into playbooks even easier. The release of the SecretHub Ansible plugin is planned for Q3 2018. Feel free to drop us a line if you want it sooner.

Happy coding!

P.S. if anything is unclear or needs changing please reach out or create a pull request on the ansible-examples repository.