Red Hat OpenShift Administration III: Scaling Deployments in the Enterprise

When you create an SA in OpenShift, the SA automatically contains two secrets: an API token and credentials for the OpenShift Container Registry. The automatically generated API token and credentials never expire. However, you can revoke the token and credentials by deleting the secrets, because OpenShift automatically generates a new secret.

Starting with Kubernetes 1.23, SAs do not automatically create the secrets that contain long-term credentials for accessing the Kubernetes API. Thus, you must obtain the credentials by using the TokenRequest API. The tokens that the TokenRequest API provides are more secure than the tokens that are stored in secrets, because the TokenRequest tokens have a bounded lifetime; other API clients cannot read the TokenRequest tokens; and OpenShift automatically invalidates the TokenRequest tokens when the pod that they are mounted into is deleted.

Although Red Hat OpenShift Container Platform (RHOCP) 4.14 is based on Kubernetes 1.27, it still creates the SA token secrets to communicate with the Kubernetes API server, because some features and workloads need the SA secrets. However, this behavior will change in a future release. Thus, although you can use the automatically generated token secret to authenticate the SA to the Kubernetes API server, Red Hat recommends using the TokenRequest API to generate tokens that are bound to the SA, and not relying on the automatically generated token secret.

The following diagram summarizes the authentication flow by using the TokenRequest API:

For external applications, you can manually generate a bound SA token through the TokenRequest API, and use it as the bearer token for authentication to the Kubernetes API. Generating a bound SA token in OpenShift by using the TokenRequest API does not remove or override the automatically generated token secret.

After creating the SA, use the following command to generate a bound SA token by using the TokenRequest API:

[user@host ~]# oc create token SA_name -n project

You can set the lifetime for the SA token by using the --duration option. The default lifetime for the SA token is one hour. You must manually refresh the manually generated SA token before it expires, so that your application can continue authenticating to the Kubernetes API.

The following example shows a decoded JWT SA token:

HEADER
...output omitted...
PAYLOAD:DATA
{
  "aud": [
    "https://kubernetes.default.svc"
  ],
  "exp": 1698059299, 
  "iat": 1698055699, 
  "iss": "https://kubernetes.default.svc",
  "kubernetes.io": {
    "namespace": "my-project", 
    "serviceaccount": {
      "name": "my-sa", 
      "uid": "3acb4630-babe-4e85-996d-cbc80f62146f"
    }
  },
  "nbf": 1698055699,
  "sub": "system:serviceaccount:my-project:my-sa"
}
SIGNATURE
...output omitted...

OpenShift assigns an SA to every pod that you deploy. By default, OpenShift creates the following SAs for every project:

You can manage SAs by using the typical oc commands, such as create, get, or describe. You can also grant roles to SAs the same as for regular users.

If you grant roles to an SA, then you must use the name of the SA together with the name of the project and the system:serviceaccount string, according to the following syntax:

system:serviceaccount:<project>:<name>

You can also use the -z option to avoid using the long system:serviceaccount:project:sa_name name, and use instead the short sa_name name.

Whenever you create an SA, it is automatically a member of the following groups:

OpenShift assigns the username and the groups for the user by using the common name (CN) and the organization (O) fields from the certificate, respectively. You can use role-based access control (RBAC) rules to provide the minimal rights to the client account to perform the job. For example, you could provide view permissions for the entire cluster to a monitoring tool, or edit permissions on selected projects to a CI/CD pipeline.

To create client certificates by using the internal OpenShift CA, you must follow these steps:

When you run any oc command, OpenShift reads a kubeconfig file in the following ways in turn:

When you log in to the OpenShift cluster through the oc login command for the first time, OpenShift creates a kubeconfig file in the default location at ~/.kube/config, if the file does not exist. You can add more authentication and connection details to the kubeconfig file automatically by using the oc login and oc project commands, or by manually editing the kubeconfig file.

You can read the details from your kubeconfig file by using the oc config view command or by opening the file with a text editor. The following example shows the parameters for a kubeconfig file from an OpenShift cluster:

apiVersion: v1
clusters: 
- cluster:
    server: https://api.ocp4.prod.com:6443
  name: production
- cluster:
    server: https://api.ocp4.stage.com:6443
    certificate-authority: ocp-apiserver-cert.crt
  name: stage
users: 
- name: admin-production
  user:
    token: REDACTED
- name: admin-stage
  user:
    client-certificate: admin-stage.crt
    client-key: tls.key
contexts: 
- context:
    cluster: production
    namespace: prod-app
    user: admin-production
  name: prod-app/api-ocp4-prod-com:6443/admin-production
- context:
    cluster: stage
    namespace: demo-app
    user: admin-stage
  name: demo-app/api-ocp4-stage-com:6443/admin-stage
current-context: demo-app/api-ocp4-stage-com:6443/admin-stage
kind: Config
preferences: {}

Although you can set most kubeconfig file parameters by using the oc login and oc project OpenShift commands, you can use the oc config command to manually configure these parameters if needed, instead of directly modifying the file. You can use the oc config command with the --kubeconfig=file option to create and store kubeconfig files with different parameters that you can use when required. The oc config command comes from the Kubernetes kubectl CLI tool, with no modifications from OpenShift.

Use the following oc config subcommands to manually configure your kubeconfig files:

As a system administrator, use the --as and --as-group options when using the oc command to impersonate a user or group. Use the oc auth can-i command to test the user access to a particular resource in the cluster. Use the -n option to specify a project for the request, or use the -A option to verify the action in all the projects. Thus, use the following command to test the RBAC rules for a particular user or group by impersonating them:

[user@host ~]$ oc auth can-i command --as user_to_impersonate \
  --as-group group_to_impersonate

You can also list all the permissions for a specific user or group by using the oc auth can-i --list command.

To grant a regular user permissions to impersonate another user, you must create a custom role with the appropriate permissions, and then create a role binding to assign the role to the user.

For example, to allow a regular user to run commands as an administrator, you can create the following cluster role, which enables anyone to impersonate the administrator user:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: sudo-admin
rules:
- apiGroups: [""]
  resources: ["users"]
  verbs: ["impersonate"]
  resourceNames: ["admin"]

Then, bind the role to the user:

[user@host ~]$ oc create clusterrolebinding binding_name \
  --clusterrole sudo-admin --user regularuser__

The user can impersonate the admin user by using the --as=admin option. The following example shows how the user cannot retrieve the node information when using their account, but can retrieve that information when impersonating the admin user:

[user@host ~]$ oc get nodes
Error from server (Forbidden): nodes is forbidden: User "regularuser" cannot list resource "nodes" in API group "" at the cluster scope
[user@host ~]$ oc get nodes --as admin
NAME       STATUS   ROLES                  AGE   VERSION
master01   Ready    control-plane,master   62d   v1.25.7+eab9cc9
worker01   Ready    worker                 57d   v1.25.7+eab9cc9