DO280 - ch05s04

Bookmark this page

Guided Exercise: Multus Secondary Networks

Expose a PostgreSQL database to external access by using a secondary network.

Outcomes

Make a PostgreSQL database accessible outside the cluster on an isolated network by using an existing node network interface.

As the student user on the workstation machine, use the lab command to prepare your system for this exercise.

This command ensures that the environment is ready.

[student@workstation ~]$ lab start non-http-multus

Instructions

Deploy a sample database.
1. Log in to the OpenShift cluster as the developer user with the developer password.
```
[student@workstation ~]$ oc login -u developer -p developer \
  https://api.ocp4.example.com:6443
Login successful.

...output omitted...
```
2. Create a non-http-multus project.
```
[student@workstation ~]$ oc new-project non-http-multus
...output omitted...
```
3. Create the resources that the ~/DO280/labs/non-http-multus/deployment.yaml file contains.
```
[student@workstation ~]$ oc apply -f ~/DO280/labs/non-http-multus/deployment.yaml
secret/database created
persistentvolumeclaim/database created
deployment.apps/database created
```
4. Wait until all resources are ready.
```
[student@workstation ~]$ oc get all
NAME                            READY   STATUS    RESTARTS   AGE
pod/database-654db5f958-8p6m5   1/1     Running   0          3m36s

NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/database   1/1     1            1           3m36s

NAME                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/database-654db5f958   1         1         1       3m36s
```
  This application contains only a deployment, a persistent volume claim, and a secret. The application does not contain any services, so the database is not accessible outside the pod network.
  This application uses a database that requires exclusive access to the database data. On the database deployment, only one pod must be running at a time. To prevent multiple pods from running at a time, the deployment uses the recreate strategy.
  This scenario is part of the scenarios where you assign a network interface exclusively to a pod. In these scenarios, the host device strategy is suitable. A network attachment with the host device strategy is suitable only for a single pod.
  In other scenarios, you must use more complex network attachments.

Examine the cluster nodes and inspect the network interface that you use in this exercise.

[student@workstation ~]$ oc login -u admin -p redhatocp \
  https://api.ocp4.example.com:6443
Login successful.

...output omitted...

Use the oc get node command to list the cluster nodes.

[student@workstation ~]$ oc get node
NAME       STATUS   ROLES                         AGE   VERSION
master01   Ready    control-plane,master,worker   36d   v1.27.6+f67aeb3

The cluster has a single node with the control plane and worker roles.

Run the ip addr command in the node, by using the oc debug command to execute commands in the node.

[student@workstation ~]$ oc debug node/master01 -- chroot /host ip addr
Temporary namespace openshift-debug-mrchh is created for debugging node...
Starting pod/master01-debug ...
To use host binaries, run: chroot /host
Pod IP: 192.168.50.10
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
...output omitted...
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UP group default qlen 1000  
    link/ether 52:54:00:00:32:0a brd ff:ff:ff:ff:ff:ff
3: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000  
    link/ether 52:54:00:01:33:0a brd ff:ff:ff:ff:ff:ff
    inet 192.168.51.10/24 brd 192.168.51.255 scope global dynamic noprefixroute ens4
       valid_lft 461179517sec preferred_lft 461179517sec
    inet6 fe80::b9dd:9436:4fc7:738/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 22:31:45:7a:e2:e3 brd ff:ff:ff:ff:ff:ff
5: ovn-k8s-mp0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN group default qlen 1000
...output omitted...
6: br-int: <BROADCAST,MULTICAST> mtu 1400 qdisc noop state DOWN group default qlen 1000
    link/ether 52:db:28:19:51:94 brd ff:ff:ff:ff:ff:ff
8: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 52:54:00:00:32:0a brd ff:ff:ff:ff:ff:ff
    inet 192.168.50.10/24 brd 192.168.50.255 scope global dynamic noprefixroute br-ex
...output omitted...

	The `ens3` interface is the main network interface of the cluster.
	The `ens4` interface is an additional network interface for exercises that require an additional network. This interface is attached to a 192.168.51.0/24 network, with the 192.168.51.10 IP address.

The system has other interfaces, including bridges and pod network interfaces.

Examine the networking configuration of the workstation machine. The workstation machine has no access to the 192.168.51.0/24 network, which is the ens4 interface in the cluster node.

Use the ip addr command to examine the network interfaces in the workstation machine.

[student@workstation ~]$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000  
    link/ether 52:54:00:00:fa:09 brd ff:ff:ff:ff:ff:ff
    inet 172.25.250.9/24 brd 172.25.250.255 scope global noprefixroute eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe00:fa09/64 scope link
       valid_lft forever preferred_lft forever
...output omitted...

The workstation machine has a single Ethernet interface. This interface is on a different network from the ens4 interface in the cluster node.

Use the route command to view the routing table in the workstation machine.

[student@workstation ~]$ ip route
default via 172.25.250.254 dev eth0 proto static metric 100
10.88.0.0/16 dev podman0 proto kernel scope link src 10.88.0.1
172.25.250.0/24 dev eth0 proto kernel scope link src 172.25.250.9 metric 100
192.168.50.0/24 via 172.25.250.253 dev eth0 proto static metric 100

The workstation routing table does not have a route to the 192.168.51.0/24 network.

Use the ping command to check connectivity to the ens4 interface in the cluster.
```
[student@workstation ~]$ ping 192.168.51.10
PING 192.168.51.10 (192.168.51.10) 56(84) bytes of data.
^C
--- 192.168.51.10 ping statistics ---
6 packets transmitted, 0 received, 100% packet loss, time 5137ms
```
The command does not produce any output after printing the first line. Wait a few seconds, and then press Ctrl+C to interrupt the ping command. The ping command prints that after transmitting some packets, no response is received.
The workstation machine cannot connect to the additional cluster network.

Examine the networking configuration of the utility machine. The utility machine has access to the 192.168.51.0/24 network.

Use the ssh command to connect to the utility machine.

[student@workstation ~]$ ssh utility
...output omitted...
[student@utility ~]$

Use the ip addr command to examine the network interfaces in the utility machine.

[student@utility ~]$ ip addr
...output omitted...
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000  
    link/ether 52:54:00:02:33:fe brd ff:ff:ff:ff:ff:ff
    inet 192.168.51.254/24 brd 192.168.51.255 scope global noprefixroute eth2
...output omitted...

The eth2 interface is attached to the 192.168.51.0/24 network, with the 192.168.51.254 IP address.

Use the ping command to check connectivity to the ens4 interface in the cluster.

[student@utility ~]$ ping 192.168.51.10
PING 192.168.51.10 (192.168.51.10) 56(84) bytes of data.
64 bytes from 192.168.51.10: icmp_seq=1 ttl=64 time=0.687 ms
64 bytes from 192.168.51.10: icmp_seq=2 ttl=64 time=0.169 ms
^C
--- 192.168.51.10 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1058ms
rtt min/avg/max/mdev = 0.169/0.428/0.687/0.259 ms

Wait a few seconds, and then press Ctrl+C to interrupt the ping command. The ping command shows that the utility machine can connect to the additional cluster network.

Exit the SSH session to go back to the workstation machine.

[student@utility ~]$ exit
logout
Connection to utility closed.
[student@workstation ~]$

Configure a network attachment definition for the ens4 interface, so that the custom network can be attached to a pod.

Edit the ~/DO280/labs/non-http-multus/network-attachment-definition.yaml file. Use the custom name, the host-device type, and the ens4 device. Configure IP address management to use the static type, with the 192.168.51.10/24 address.

apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: custom
spec:
  config: |-
    {
      "cniVersion": "0.3.1",
      "name": "custom",
      "type": "host-device",
      "device": "ens4",
      "ipam": {
        "type": "static",
        "addresses": [
          {"address": "192.168.51.10/24"}
        ]
      }
    }

Use the diff command to compare your network attachment definition with the solution in the ~/DO280/solutions/non-http-multus/network-attachment-definition.yaml file. If the files are identical, then the diff command does not return any output.
```
[student@workstation ~]$ diff \
  ~/DO280/labs/non-http-multus/network-attachment-definition.yaml \
  ~/DO280/solutions/non-http-multus/network-attachment-definition.yaml
```

Use the oc create command to create the network attachment definition.

[student@workstation ~]$ oc create \
  -f ~/DO280/labs/non-http-multus/network-attachment-definition.yaml
networkattachmentdefinition.k8s.cni.cncf.io/custom created

Edit the deployment to add the k8s.v1.cni.cncf.io/networks annotation with the custom value.

[student@workstation ~]$ oc login -u developer -p developer \
  https://api.ocp4.example.com:6443
Login successful.

...output omitted...

Edit the ~/DO280/labs/non-http-multus/deployment.yaml file to add the k8s.v1.cni.cncf.io/networks annotation with the custom value.

apiVersion: v1
...output omitted...
- apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: database
  spec:
    replicas: 1
    strategy:
      type: Recreate
    selector:
      matchLabels:
        name: database
        app: database
    template:
      metadata:
        labels:
          name: database
          app: database
        annotations:
          k8s.v1.cni.cncf.io/networks: custom
      spec:
...output omitted...

Use the oc apply command to add the annotation.

[student@workstation ~]$ oc apply -f ~/DO280/labs/non-http-multus/deployment.yaml
secret/database configured
persistentvolumeclaim/database unchanged
deployment.apps/database configured

Wait until all resources are ready.

[student@workstation ~]$ oc get all
NAME                            READY   STATUS    RESTARTS   AGE
pod/database-74d79685f7-8p6m5   1/1     Running   0          3m36s

NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/database   1/1     1            1           3m36s

NAME                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/database-654db5f958   0         0         0       15m
replicaset.apps/database-74d79685f7   1         1         1       3m36s

Examine the k8s.v1.cni.cncf.io/network-status annotation in the pod.

[student@workstation ~]$ oc get pod database-74d79685f7-6schp \
  -o jsonpath='{.metadata.annotations.k8s\.v1\.cni\.cncf\.io/network-status}'
[{
    "name": "ovn-kubernetes",
    "interface": "eth0",
    "ips": [
        "10.8.0.92"
    ],
    "mac": "0a:58:0a:08:00:5c",
    "default": true,
    "dns": {}
},{
    "name": "non-http-multus/custom",
    "interface": "net1",
    "ips": [
        "192.168.51.10"
    ],
    "mac": "52:54:00:01:33:0a",
    "dns": {}
}]

Note

The period is the JSONPath field access operator. Normally, you use the period to access parts of the resource, such as in the .metadata.annotations JSONPath expression. To access fields that contain periods with JSONPath, you must escape the periods with a backslash (\).

Verify that you can access the database from the utility machine.

Use the ssh command to connect to the utility machine.

[student@workstation ~]$ ssh utility
...output omitted...
[student@utility ~]$

[student@utility ~]$ podman login --tls-verify=false \
  registry.ocp4.example.com:8443 -u developer -p developer
Login Succeeded!

Run a command to execute a query on the database. Use the IP address on the custom network to connect to the database. Use password as the password for the user.

[student@utility ~]$ podman run -it --tls-verify=false \
  --entrypoint=/usr/bin/psql \
  registry.ocp4.example.com:8443/rhel8/postgresql-13:1-7 \
  -h 192.168.51.10 -U user sample -c 'SELECT 1;'
...output omitted...
Password for user user: password
 ?column?
----------
        1
(1 row)

Exit the SSH session to return to the workstation machine.

[student@utility ~]$ exit
logout
Connection to utility closed.
[student@workstation ~]$

Verify that you cannot use the same process to access the database from the workstation machine, because the workstation machine cannot access the custom network.
1. Log in to the classroom container registry for access to an image with database utilities.
```
[student@workstation ~]$ podman login --tls-verify=false \
  registry.ocp4.example.com:8443 -u developer -p developer
Login Succeeded!
```
2. Run a command to execute a query on the database. Use the IP address on the custom network to connect to the database.
```
[student@workstation ~]$ podman run -it --tls-verify=false \
  --entrypoint=/usr/bin/psql \
  registry.ocp4.example.com:8443/rhel8/postgresql-13:1-7 \
  -h 192.168.51.10 -U user sample -c 'SELECT 1;'
...output omitted...
Storing signatures
psql: error: could not connect to server: Connection refused
	Is the server running on host "192.168.51.10" and accepting
	TCP/IP connections on port 5432?
```
  After the image is downloaded, the command pauses for over a minute, because you cannot access the custom network from the workstation machine.
  The deployment uses the custom network, and you can access the database only through the custom network.

Finish

On the workstation machine, use the lab command to complete this exercise. This step is important to ensure that resources from previous exercises do not impact upcoming exercises.

[student@workstation ~]$ lab finish non-http-multus

Discuss Red Hat OpenShift Administration II: Configuring a Production Cluster

Go to community

Red Hat OpenShift Administration II: Operating a Production Kubernetes Cluster (DO280)

Haley_Ruccio

26 lip 2023

Configure, manage, and troubleshoot OpenShift clusters and containerized applicationsRed Hat OpenShift Administration II: Operating a Production Kubernetes Cluster (DO280) prepares OpenShift Cluster Administrators to perform daily administration tasks on clusters that host applications provided by internal teams and external vendors, enable self-service for cluster users with different roles, and deploy applications that require special permissions such as such as CI/CD tooling, performance monitoring, and security scanners. This course focuses on configuring multi-tenancy and security features of OpenShift as well as managing OpenShift add-ons based on operators.The skills you learn in this course can be applied using all versions of OpenShift, including Red Hat OpenShift on AWS (ROSA), Azure Red Hat OpenShift, and OpenShift Container Platform.This course is based on OpenShift Container Platform 4.12.5-10 Course TopicsDeploying packaged applications using manifests, templates, kustomize, and helm.Configuring authentication and authorization for users and applications.Protecting network traffic with network policies and exposing applications with proper network access.Deploying and managing applications using resources manifests.Enabling developer self-service of application projects.Managing OpenShift cluster updates and Kubernetes operator updates.Target AudienceSystem Administrators and Platform Operators interested in the ongoing management of OpenShift clusters, applications, users, and add-ons.Site Reliability Engineers interested in the ongoing maintenance and troubleshooting of Kubernetes clusters.System and Software Architects interested in understanding the security of an OpenShift cluster.Recommended TrainingTake our free assessment to gauge whether this offering is the best fit for your skills.Prerequisite: Red Hat OpenShift I: Containers & Kubernetes (DO180 v4.12), or equivalent skills deploying and managing Kubernetes applications using the OpenShift web console and command-line interfaces.Technology considerationsThis course requires internet access to access the cloud-based classroom environment that provides an OpenShift cluster and a remote administrator’s workstation.

476

About pod security standards and warnings

alexcorcoles

17 kwi 2023

(This material has been created with the inputs of instructors and other people. We are trying to get this material published as soon as possible to address some concerns with DO180 and DO280.)With the default configuration, any namespace that you create on OpenShift 4.12 has the following labels: pod-security.kubernetes.io/audit: restrictedpod-security.kubernetes.io/audit-version: v1.24pod-security.kubernetes.io/warn: restrictedpod-security.kubernetes.io/warn-version: v1.24 These labels activate auditing and warning of the restricted pod security standard. The Kubernetes documentation describes the restricted pod security standard. Pod security standards such as restricted are not directly related to security context constraints such as the restricted SCC.Kubernetes uses an admission controller to reject pods that do not comply with pod security standards. An admission controller is a Kubernetes component that inspects every Kubernetes resource before its creation, and acts before Kubernetes creates the resource. This action can include rejecting the creation of the resource, or modifying the resource.The pod security standards are policies that include properties that workloads must follow. For example, the restricted standard requires that containers set the securityContext.allowPrivilegeEscalation field to false.Pod security standards apply both to workloads, such as deployments and jobs, and to pods.When a pod is created, the admission controller inspects the container specifications and enumerates the fields that violate the pod security standard of the namespace. The admission controller inspects the pod-security labels in the namespace to decide which pod security standard to apply (restricted, baseline, or privileged), and which action to take on violations. Namespaces can audit, warn, or enforce pod security standard violations. When the pod-security.kubernetes.io/audit annotation is set to restricted, the admission controller prevents the creation of pods that violate the security standard. When the pod-security.kubernetes.io/audit and pod-security.kubernetes.io/warn annotations are defined in the project, the admission controller does not block the creation of pods, but registers audit events or warnings.The admission controller does not enforce pod security standards for workloads. If the namespace is configured to enforce a pod security standard, then you can create a violating workload. The admission controller enforces only pod security policies on pods.Cluster administrators can use pod security standards to ensure that pods and workloads in namespaces limit their privileges. By limiting privileges, cluster administrators can mitigate what malicious pods can do.The default configuration of user namespaces in OpenShift does not restrict any workload; it only notifies users when they create workloads that do not follow pod security standards that might be enforced in future OpenShift versions.Security Context ConstraintsOpenShift introduced security context constraints (SCC) before the introduction of pod security standards in Kubernetes. SCCs automatically make security modifications to pods when they are created. By default, the system:openshift:scc:restricted-v2 cluster role binding applies to all authenticated users, so regular user workloads use the restricted-v2 SCC. For example, when you execute the following command to create a workload without any security configuration, the SCC adds security configuration to the pods: [user@host ~]$ oc create deployment test --image registry.ocp4.example.com:8443/redhattraining/hello-world-nginx (You can add the --dry-run=client -o yaml options to view the deployment manifest. The deployment manifest does not contain the security properties, which are described later.)If you inspect the pods that the deployment created, the SCC makes the following changes: [user@host ~]$ oc get pod test-b8d5658b6-7bxfs -o yamlapiVersion: v1kind: Podmetadata: annotations:... output omitted... openshift.io/scc: restricted-v2 seccomp.security.alpha.kubernetes.io/pod: runtime/default...output omitted...spec: containers: - image: registry.ocp4.example.com:8443/redhattraining/hello-world-nginx imagePullPolicy: Always name: hello-world-nginx resources: {} securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL runAsNonRoot: true runAsUser: 1000690000... output omitted... securityContext: fsGroup: 1000690000 seLinuxOptions: level: s0:c26,c20 seccompProfile: type: RuntimeDefault... output omitted... These restrictions satisfy the restricted pod security policy.The result of these defaults means that when creating a workload (such as a deployment) in a namespace without any customization, the restricted pod security standard applies, but the admission controller only warns and audits. If the workload does not satisfy the pod security standard, then the admission controller allows the workload to be created, but prints a message such as the following warning: Warning: would violate PodSecurity "restricted:v1.24": allowPrivilegeEscalation != false (container "example" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "example" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "database" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "example" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost") However, by default Kubernetes creates the deployment pods with the restricted-v2 SCC, with the restrictions that the previous warning included. Due to the following factors, workloads generate only a warning, and the pod security standard admission controller allows their pods:The pod security standard admission controller never enforces pod security standards on workloads (only on pods).By default, namespaces do not enforce pod security standards on pods.By default, regular workloads create pods by using the restricted-v2 SCC, which satisfies the restricted pod security standard.Even if future versions of OpenShift change the default namespace configuration to enforce the restricted pod security standard, users will continue to be able to create workloads without explicitly declaring the restrictions that the pod security standard requires, because the restricted-v2 SCC will apply such restrictions automatically.You can take the following approaches to handling pod security:For most workloads, the restricted-v2 SCC limits pod privileges adequately. When creating a workload, you receive the warning, but the workload creates pods and works correctly.You can also create your workloads by providing a full specification of the security constraints to apply, without relying on the SCC to provide the configuration. By following this approach, privileges are more explicit, and you have greater control, so you can adjust more precisely the specific privileges that are granted to workloads.When the restricted-v2 SCC and the default restricted pod security standard are not adequate for your workload, because either you require further privileges, or you need further restrictions, you must use an existing SCC, or create a custom SCC, and provide explicit security specifications. You can also adjust the namespace labels to choose a different pod security standard, and decide to audit, log, or enforce the standard.

1473

Welcome to the Red Hat OpenShift Administration II (DO280) group in the Red Hat Learning Community!

Deanna

17 kwi 2023

We are excited to launch a space dedicated to the Red Hat Training course Red Hat OpenShift Administration II: Operating a Production Kubernetes Cluster!To gain the most value from this group - click the "Join Group" button in the upper right hand corner.We encourage group members to collaborate in this group to discuss topics, ask questions, share best practices and tips, provide course feedback, and share their accomplishments as it relates to DO280.Read more about Red Hat OpenShift Administration II here.Thanks!

1334

Revision: do280-4.14-08d11e1