Certificates in Containers
Preface
Everything below assumes that you have created the necessary DNS-records according to instructions.
ACME is a protocol, which is used by cert-manager to renew SSL-certificates automatically. This is enabled by the cert-manager -operator, which is provided to us by a Red Hat operator. The cert-manager operator uses an Issuer-object to define a connection to a certificate backend, and together with Ingress and/or Certificate objects retrieves the certificate and injects it into a Route. This instruction will focus on using Let’s Encrypt as our certificate authority. More information can be found in the general usage docs of the cert-manager operator.
Quicklinks:
We also have a helper page for creating Ingress and Issuer files, you can find it here: prod, test
Let’s Encrypt
Let’s Encrypt is a non-profit certificate organization, that is free to use. During testing you should connect to the staging environment, which doesn’t have strict rate-limits. After your configurations are working, you can switch to using the production server. Let’s Encrypt also requires your website to be accessible from the Internet, so the label type: external is required. This instructions uses Ingress resources, and each Ingress will create a Route object which works like normal. This is because Ingress is the vanilla kubernetes resource, so the hooks that the cert-manager operator has to it are more developed.
| Server | url | Certificate validity | Rate limits |
|---|---|---|---|
| prod | https://acme-v02.api.letsencrypt.org/directory | Your browser trusts the certificate. | Strict |
| staging | https://acme-staging-v02.api.letsencrypt.org/directory | Your browser claims that the certificate is not valid, which is intended behaviour. You can still check that the certificate is added to your page by using your browser, and it will show that the Issuer is Let's Encrypt (staging). | Relaxed |
Issuer
We recommend creating an Issuer for both the staging and the production server of Let’s Encrypt. The provided files are ready to go, as long as you add your email and namespace to the correct lines marked with <text>. By changing the value in spec.acme.server you can decide which certificate issuer is targeted.
kind: Issuer
metadata:
name: letsencrypt-staging
namespace: <namespace>
spec:
acme:
email: <your-email-here>
privateKeySecretRef:
name: letsencrypt-staging-acme-key
server: 'https://acme-staging-v02.api.letsencrypt.org/directory'
solvers:
- http01:
ingress:
ingressClassName: openshift-default
ingressTemplate:
metadata:
labels:
type: external
letsencrypt-staging.issuer.yaml
Unlike the instructions provided by OpenShift documentation, we need to add the following part to the solvers part of the Issuer. This is because by default, all Routes are created using the *.apps Ingress, which is not accessible from the internet in Tike clusters. By adding this, the temporary Ingress that Let’s Encrypt uses to verify the service gets the correct labels and the certificate-issuing process receives a valid response (200 instead of 503).
metadata:
labels:
type: external
It is also possible to use a DNS-challenge, which is necessary when you service is not accessible from the Internet (e.g. the CNAME of your service is host.apps.ocp-prod-0...).
Ingress
Ingress vs Route
Both of the following resources achieve the same thing, the difference being that Ingress objects can reference Secrets to pull the certificates from, and Routes need to have the certificates and keys in plain text.
| Ingress | Route |
|---|---|
| metadata: labels: type: external spec: ingressClassName: openshift-default tls: - hosts: - my-service.domain.fi secretName: <name-for-tls-secret-for-my-url> rules: - host: my-service.domain.fi http: paths: - path: / pathType: Prefix backend: service: name: frontend port: number: 5173 | metadata: labels: type: external spec: host: my-service.domain.fi path: / to: kind: Service name: frontend weight: 100 port: targetPort: 5173-tcp tls: termination: edge certificate: #removed key: #removed insecureEdgeTerminationPolicy: Redirect wildcardPolicy: None |
Thankfully we can use cert-manager to create the certificate by asking for one in the Ingress, which means that the certificate information will be injected to the Route that is automatically generated.
Creating the Ingress
We recommend naming the Ingress my-service-domain-fi or something similar for easy identification.
We also recommend creating the Ingress without metadata.annotations or spec.tls to first make sure that the Ingress is created correctly. By removing the tls and annotation, the created Route will direct any traffic to an unencrypted address http://my-service.domain.fi
After making sure the Ingress works, remember to first use the Issuer called letsencrypt-staging, so that you don’t run into rate-limiting from Let’s Encrypt. After getting a working certificate, change to the correct Issuer. If the certificate’s issuer doesn’t update in a reasonable time, either change the name of the Secret or delete the old one. This will force a refresh.
apiVersion: networking.k8s.io/v1
metadata:
annotations:
cert-manager.io/issuer: letsencrypt
name: my-service.domain.fi
namespace: <namespace>
labels:
type: external
spec:
ingressClassName: openshift-default
tls:
- hosts:
- my-service.domain.fi
secretName: <name-for-tls-secret-for-my-url>
rules:
- host: my-service.domain.fi
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: frontend
port:
number: 5173
Even though the cert-manager docs say that the annotation cert-manager.io/common-name: my-domain.fi is optional, there have been instances where either the Certicate or CertificateRequest -objects were not created and the SSL-certificate didn’t work. You can try adding the annotation to the Ingress, and then follow steps in the troubleshooting part.
Cert-manager needs the metadata.annotations.cert-manager.io/issuer: <issuer name> to work properly. Again match this to the name of the Issuer you created.
The spec.tls part defines the hostname and which certificate to use for it, make sure that it matches the spec.secretName defined in the Certificate. This Ingress will create a Route that is accessible from the internet, if you wish to create one only accessible via University internal networks and VPN, remove the label type: external
labels:
type: external
Troubleshooting ACME
If you keep having issues with the certificates, check them by searching for certificates in the OpenShift web-console or with the command
oc describe certificate <certificate-name> -n <namespace>
For more information about the Ingress resource, check the part Ingress vs Route.
If the certificates don’t appear, you can check the Issuer to see if it has made a connection to the ACME backend. The Issuer should be ready and the status should be The ACME account was registered with the ACME server.
NAME READY STATUS AGE
acme-issuer True The ACME account was registered with the ACME server 19d
If the Issuer is fine, you can try deleting the Secret mentioned in the Issuer
apiVersion: networking.k8s.io/v1
metadata:
annotations:
cert-manager.io/common-name: my-domain.fi
cert-manager.io/issuer: letsencrypt
spec:
ingressClassName: openshift-default
tls:
- hosts:
- my-domain.fi
secretName: my-domain-fi-tls
In this case, we would delete the Secret my-domain-fi-tls.