Secrets

This page is under construction. You will find info about Secrets CSI driver, SealedSecrets, External Secrets and other possible solutions here.

Types of Secrets

There are multiple types of Secrets, but this instruction set will focus on Opaque, or key-value pair Secrets. More info in Kubernetes docs.

Creating a Secret

There are 3 main ways of creating a Secret.

1. Creating one from a yaml file, where each key: value pair is placed under stringData. If you wish to base64 encode the values for yourself, switch out stringData for data.

you can then use oc apply -f <filename> or add it from the +-symbol of the OpenShift UI.

1. From the terminal with the command oc create secret

More options and info in Kubernetes docs

1. From the UI

Navigate to the Secrets page and then click Create and choose your type of secret. The UI will guide you through the rest.

• Developer -> Secrets
• Administrator -> Workloads -> Secrets

NOTE: Secrets are, as the name implies, to be kept secret. Do not put them in version control etc.

Checking the contents of a Secret

From the command line

You can retrieve the base64 encoded values for all keys with the oc get command, and then you can decode them by using echo and base64.

The Kubernetes docs mention a way of inline decoding a single key’s value, but since it is not recommended we won’t list it here.

Retrieving a Secret

You can retrieve the Secret, either from the UI or from the command line. We recommend using the oc get command, since the UI will add a lot of unnecessary information (unless you only copy what is necessary).

Retrieving a Secret (cli)

You can retrieve a Secret, just like any Kubernetes object by using the oc get <kind> <name> -oyaml. You can also output it directly into a file by adding > filename.yaml at the end of the command. Remember that since the uid and creationTimeStamp are related to this specific object in this specific cluster, you should delete them.

Retrieving a Secret (UI)

From the UI

You can check out Secret objects from both the Developer and the Administrator perspectives, but the path changes slightly. You can either reveal all values, copy a single one from the UI without revealing it or show the yaml that defines the Secret.

• Developer -> Secrets -> my-secret
• Administrator -> Workloads -> Secrets -> my-secret

Revealing the values and copying a single value without revealing it happen from the Details tab of the Secret, and the yaml is found on the YAML tab. The OpenShift UI add a lot of information under metadata, but we only need the name, namespace and possible labels or annotations for it.

Editing a Secret

You can either edit the Secret in the cluster with oc edit secret mysecret, edit a local copy and use oc apply -f <filename> or use Actions -> Edit Secret from the OpenShift UI.

SealedSecrets

Since SealedSecrets is already available in Tike Container platform, we will start with it. It consist of two parts, the operator running in the cluster that is provided by container administration, and a client side utility called kubeseal. The idea is to encrypt the contents of a Secret objects against a sealing key that is managed and automatically renewed by the operator in each cluster, so that the created SealedSecret object can be placed in version control.

By default, the Secret is encrypted with knowledge of the correct namespace, which means that the the encrypted definition can’t be reused as-is in another namespace. This is to help against other users of the same cluster getting access to your Secrets.

• The kubeseal utility uses asymmetric crypto to encrypt secrets that only the controller can decrypt. Source

NOTE 1: Always download the most recent version of the kubeseal binary using the instructions on the Github page and notify container administration if there are issues related to version numbers between the kubeseal binary and the sealed-secrets operator running in the cluster.

NOTE 2: The sealed-secrets documentation mentions, that the sealing key is renewed every 30 days, but since encryption happens outside the cluster, old keys will not be deleted. This lessens the impact of a single sealing-key leaking, but is not equal to key rotation.

NOTE 3: As stated before, SealedSecrets are encrypted and therefore can be set in version control, unlike Secrets.

Using Sealed Secrets

1. Acquire a Secret, either by making one through the OpenShift UI or by using oc/kubectl commands. For instructions, check Creating a Secret from above. The Sealed Secrets Github page has more comprehensive instructions. If you copy the Secret from inside OpenShift, we recommend clearing the bits of metadata you don’t need, only leaving name, namespace and the necessary parts under labels or annotations. Remember that you need to use stringData for plaintext values and data for base64 encoded values, so that the Sealed Secret operator knows how to decrypt the Secret
   kind: Secret
   apiVersion: v1
   metadata:
    name: example-secret
    namespace: dokumentaatio
   data:
    password: cGxhY2Vob2xkZXI=
    username: cGxhY2Vob2xkZXI=
   type: Opaque
2. Login to the correct cluster with oc. Make a local file and use the kubeseal command against it.
   kubeseal -f <filename> -w <file-output-name>
   e.g.
   kubeseal -f secret.yaml -w sealedsecret.yaml

This will create a file called sealedsecret.yaml that looks a little something like this. As you can see, the created SealedSecret has the same name as the Secret it was created from.

1. Add the SealedSecret into your version control.
2. (If no CI/CD process is in place, add the SealedSecret into OpenShift)

This can be done either with oc apply -f <filename> or by using the + symbol in the OpenShift UI.

1. Check the status of the SealedSecret.

oc describe sealedsecret <name> -n <namespace>

or by using the Search feature of the UI. For us, since the Secret already exists in the cluster, we will get this output.

oc describe sealedsecret example-secret -n dokumentaatio
...
Events:
  Type     Reason           Age                From            Message
  ----     ------           ----               ----            -------
  Warning  ErrUpdateFailed  40s (x6 over 41s)  sealed-secrets  Resource "example-secret" already exists and is not managed by SealedSecret

The “Resource already exists and is not managed by SealedSecret” can be corrected by deleting the original Secret from the cluster. After a while the SealedSecret should report

Events:
  Type    Reason    Age   From            Message
  ----    ------    ----  ----            -------
  Normal  Unsealed  3s    sealed-secrets  SealedSecret unsealed successfully

You can also check that the new Secret should have an owner reference to the SealedSecret in it’s yaml.

metadata:
  ownerReferences:
  - apiVersion: bitnami.com/v1alpha1
    controller: true
    kind: SealedSecret
    name: example-secret
    uid: 50c3d0a9-1a93-4dfd-997e-e01ed9758f9c

1. After confirming the SealedSecret works and that the contents of the Secret are correct, remember to delete the secret file from your computer.

Updating a SealedSecret

If you need to update the contents of a Secret managed by a SealedSecret, retrieve the Secret or create it manually and add the new/changed values. Retrieval instructions can be found here. Then re-run the steps from Using Sealed Secrets.

Known issues

No key could unseal

Since the Secret is encrypted against each cluster, you would need to create a separate SealedSecret for each cluster. If you run into the error “Failed to unseal: no key could decrypt secret (value1, value2..)”, you were logged in to the wrong cluster while using kubeseal or tried manually editing the namespace inside the SealedSecret object.

Events:
  Type     Reason           Age                From            Message
  ----     ------           ----               ----            -------
  Warning  ErrUnsealFailed  11s (x3 over 12s)  sealed-secrets  Failed to unseal: no key could decrypt secret (username, password)

Log in to the correct cluster, use the kubeseal command in part 2 again and then oc apply -f <filename>. The SealedSecret should be fine after this.

The Secret contents look wrong

This usually happens by manually creating the text definition of a Secret with plaintext values under data, and then using kubeseal on the file. Make sure to have the starting Secret in the correct form. More info in Creating a Secret.

I want my CI/CD solution to make SealedSecrets

Your CI/CD solution either needs a service account with permission to authenticate to the to the cluster, or you need to download the public key from the cluster. According to the Github docs, downloading the public key can be done with

kubeseal --fetch-cert >mycert.pem

and then can be used in the kubeseal command with the flag --cert

kubeseal --cert <certificate-file>

I want to change the name/namespace of the Secret

Just change the name of the Secret before using kubeseal, apply the new file and delete the old SealedSecret object.

I don’t have access to the cluster but need to decrypt a SealedSecret

Please ask the project owner to contact the container administration with the SealedSecret file and knowledge of the cluster the SealedSecret was created against. We can decrypt the Secret information, and will give it to the project owner.