HELM SECRETS 

Do you have application properties like database passwords that are not to be shared with the world? Use Helm Secrets and hide them in plain sight.

In this post we will encrypt a value and use it in a Helm deployment. The complete code can be found here on github but follow along and learn how to use this technique in all your Helm deployments.

For a more complete overview of the tool we will use go to the docs:

• Helm secrets
• sops

Prerequisites

We will need an asymmetric key pair to encrypt our secret value. The public key is used to encypt the data and only with the private key the data can be decrypted. We will use a GPG key from a previous post on GPG encryption. We could also use key management services like AWS KMS, GCP KMS or Azure Key Vault, but this is out of scope for now. With the following command you can check if you have gpg keys ready.

gpg --list-keys

To setup the key we want to use set the following environment variable with the fingerprint of the key.

export SOPS_PGP_FP="E3B52D53AFF3AE0184C5E62C5D332FB14E760DE0"

Notice the first part of the variable is SOPS, this is the tool that handles the encryption. To install it on mac you can use:

brew install sops

We are going to use an existing Helm deployment found here on github. We are going to make the current value of the slack webhook a secret.

SOPS

Get the code of the kubernetes-monitoring project and navigate to the Helm deployment directory.

cd deployment/helm/kubernetes-monitoring

Take a look at the values file for the test environment.

Now use the following sops command to create a secrets.yaml file.

sops environments/test/secrets.yaml

Now cut and paste the values you want to hide from the values file. Note that I regenerated the webhook for the old one is for ever in the git history.

prometheus:
  slack:
    webhook: https://hooks.slack.com/services/KSNDNICEJN/HDHEBECHB/isnd8sdnsdnn332

Save your changes and look at the resulting file.

At the top we still see the the key of our value in yaml format but the value itself is encrypted. And at the bottow we see our GPG key, you can compare the fingerprint.

Note that we can add multiple keys to the secrets.yaml file from multiple key stores. Every one of these keys can be used to decrypt and encrypt the values in that file.

To quickly decrypt the values you can use the sops decrypt command.

To edit existing values you can use the first sops command and the changed values will be re-encrypted when you save.

sops environments/test/secrets.yaml

In the following part we will use the Helm secrets plugin that uses the same sops functionalities we just saw.

Helm secrets

Now that we can hide our values from anyone that does not have a matching private key we want to integrate this in our deployment process. The Helm secrest plugin does excactly that. To install the plugin use the following.

helm plugin install https://github.com/futuresimple/helm-secrets

Now to use the plugin replace the helm command with helm secrets. Now you can specify the secrets file as if it were a normal values file. The plugin will search for a key to decrypt and delete the file afterwards.

helm secrets install kubernetes-monitoring .             \
                  --namespace sybrenbolandit             \
                  --values environments/test/values.yaml \
                  --values environments/test/secrets.yaml

Note that it is wise to use the secret values in Kubernetes Secrets only (kind: Secret). Otherwise everyone with read access on the cluster can retrieve the secret values there.

Hopefully you are now able to hide your secret values in an automated deployment. Happy hiding!