Skip to Content

How internet connects to pods in OpenShift

Last modified by Tommi Tuura on 2025/03/19 08:37

This page is here to explain in detail what goes on, network-wise, when a web browser or a client application connects to a service running on OpenShift cluster.

This is not exactly a how-to guide for anything, but rather an hopefully informative page to illuminate what is going on which may help you to understand: 

  1. Why you do not, in general usage, need to use ports below 1024 in your application pod.
  2. What problems you will face if you want to expose protocols other than http/https to clients running outside the openshift cluster when your service is running inside the cluster.
    • There is a way around this if the protocol your service communicates with supports TLS with SNI extension, but even then the connection must run through port 443. Instructions how to do this are provided elsewhere (coming soon...)

Do note: None of this is relevant if the connection is initiated from inside the OpenShift cluster to outside. For example, connecting from your pods to the centralized database service postgres that's running separately from openshift, should work fine just like if your service was running in its dedicated VM. (If that is not working fine for you, please contact atk-tietokannat@helsinki.fi to get your issue solved.) Also, Pods within the same OpenShift cluster (in the same namespace) can communicate directly just fine.

ingress-traffic.png

  • At the start, the web browser / client app has a hostname to connect to. As with all connections, there will be a DNS (1) query to find out the IP address where your service is accepting requests. With University of Helsinki container platform, there are essentially two IP addresses that can be used for this purpose for each OpenShift cluster. For brevity's sake we're going to refer to these as the ips as "Ext" ip and "Apps" ip. Both of these IP addresses have wildcard hostname attached to them, of form *.ext.<clustername>.k8s.it.helsinki.fi or *.apps.<clustername>.k8s.it.helsinki.fi.
    • It makes no actual difference whether the hostname your application is using is bound to the IP address via an A record directly or CNAME to some hostname that matches the wildcard. The only thing that matter are that DNS query returns the right IP address and the connection has the HTTP HOST or TLS SNI extension information in place, which will later be used by cluster router to forward your traffic to the correct pod.

    • University Container Platform management uses CNAMEs to a generic hostname host.apps/ext.clustername.k8s.it.helsinki.fi (picking correct values for apps, ext and clustername as needed) for the published applications' hostnames. Important thing to note is that this intermediate generic hostname bears no relevance to your application and you should not need to put it in any place in your yaml manifests. The hostname your application uses gets resolved into IP address and that is all your application should care about. The precise reasons why we use CNAMEs instead of direct A records are outside the scope of this document.

  • Both of these IP addresses (2) are handled by loadbalancer hosts, doubled for high availability, that will simply forward all traffic from ports 80 to 443 (and no other) to predetermined ports on certain cluster VM nodes (3). These ports are bound directly to router pods running in these cluster nodes. These router pods are the only place along this connection where a pod is directly listening to a network interface which is reachable from outside of the openshift cluster. Your application pods do not attach directly to these ports, or any ports on the cluster node's primary network interface.
  • The router pods will then do most of the work in getting the traffic in to the right place. Every single connection from outside the cluster into the cluster, across every service on the cluster that's exposed to outside connectivity, shares the same pathway through one of four ip:port combinations to these router pods. The router pod will then look at either the HTTP host header or TLS SNI extension information and find a route object in the cluster where the .spec.host value matches the correct hostname. (4) This route object will then point to a service object which should point to the pods (and ports) that will serve your application. (5) This last connection runs in the cluster internal overlay network, where the pods have their own ip addresses that are not accessible from outside the cluster.
    • These Route and Service objects are things you need to provide (in case of Route, maybe indirectly via an Ingress object) as they contain key information you need to decide for your yourself (and they are part of the yaml manifests you should include in your version control alongside Deployments and whatever else definitions are included in your project).
    • There are multiple ways to create Routes:
      • Writing or already having them as yaml files in your version-control and running oc apply on them against the cluster, just like any other of your yaml manifests with OpenShift / Kubernetes you should keep in version control alongside your sources and configurations
      • Clicking around in Web Console to create the objects either as single instances in Administor view or using a template for a complete application in the Web Console Developer view (most of which include a Route definition)
      • Using either of above methods to create an Ingress object from the networking.k8s.io/v1 API. These Ingress objects work by creating correspoding Routes under their ownership in OpenShift and chances are you actually want to use this Ingress object instead of Route. The most important improvements over Routes with Ingress is firstly, the ability to define your own certificates as Kubernetes Secret objects instead of inlining them in plaintext (including secret keys) in Route objects. And secondly, Ingress object supports the use of cert-manager operator that will automatically generate the required certificate and sign them with ACME (provided the Issuer object has been set up correctly in your project).
    • The port you listen to in you application pod and container can very well be whatever you choose above 1024 and it makes no difference from the viewpoint of your client connection, that is still connecting to the loadbalancer ip address at either port 80 or 443. Your pod and container doesn't need to (and also can not) listen to ports 80/443 on its own cluster-internal ip address that's unreachable from outside the cluster anyway. You will only need to set the correct targetPort value in your Service object.