DO280 - ch07s06

Bookmark this page

Guided Exercise: Install Operators with the CLI

Install an operator by using the command-line interface and Kubernetes manifests.

Outcomes

Install operators from the CLI with manual updates.

As the student user on the workstation machine, use the lab command to prepare your system for this exercise.

This command ensures that the cluster is ready, and removes the openshift-file-integrity namespace and File Integrity operator if they exist.

[student@workstation ~]$ lab start operators-cli

Instructions

In this exercise, you install the File Integrity operator with manual updates. The documentation of the File Integrity operator contains specific installation instructions.

For more information, refer to the Installing the File Integrity Operator Using the CLI section in the File Integrity Operator chapter in the Red Hat OpenShift Container Platform 4.14 Security and Compliance documentation at https://access.redhat.com/documentation/en-us/openshift_container_platform/4.14/html-single/security_and_compliance/index#installing-file-integrity-operator-using-cli_file-integrity-operator-installation

[student@workstation ~]$ oc login -u admin -p redhatocp \
  https://api.ocp4.example.com:6443
Login successful.

...output omitted...

Find the details of the File Integrity operator within the OpenShift package manifests.

View the available operators within the OpenShift Marketplace by using the oc get command.

[student@workstation ~]$ oc get packagemanifests
NAME                      CATALOG                           AGE
file-integrity-operator   do280 Operator Catalog Cs         37h
lvms-operator             do280 Operator Catalog Cs         37h
compliance-operator       do280 Operator Catalog Cs         37h
metallb-operator          do280 Operator Catalog Cs         37h
kubevirt-hyperconverged   do280 Operator Catalog Cs         37h

Examine the File Integrity operator package manifest by using the oc describe command.

[student@workstation ~]$ oc describe packagemanifest file-integrity-operator
Name:         file-integrity-operator
...output omitted...
Spec:
Status:
  Catalog Source:               do280-catalog-cs
  Catalog Source Display Name:  do280 Operator Catalog Cs
  Catalog Source Namespace:     openshift-marketplace
  Catalog Source Publisher:
  Channels:
...output omitted...
      Install Modes:
        Supported:  true
        Type:       OwnNamespace
        Supported:  true
        Type:       SingleNamespace
        Supported:  false
        Type:       MultiNamespace
        Supported:  true
        Type:       AllNamespaces
...output omitted...
    Name:           stable
  Default Channel:  stable
  Package Name:     file-integrity-operator
...output omitted...

The operator is in the do280-catalog-cs catalog source in the openshift-marketplace namespace. The operator has a single channel with the v1 name. The operator has the file-integrity-operator name.

Install the File Integrity operator. By following the operator installation instructions, you must install the operator in the openshift-file-integrity namespace. Also, you must make the operator available only in that namespace. The File Integrity operator requires you to create a namespace with specific labels.

The operator documentation provides a YAML definition of the required namespace. The definition is available in the ~/DO280/labs/operators-cli/namespace.yaml path. Examine the definition and create the namespace.

apiVersion: v1
kind: Namespace
metadata:
  labels:
    openshift.io/cluster-monitoring: "true"
    pod-security.kubernetes.io/enforce: privileged
  name: openshift-file-integrity

[student@workstation ~]$ oc create -f ~/DO280/labs/operators-cli/namespace.yaml
namespace/openshift-file-integrity created

Create an operator group in the operator namespace. The operator group targets the same namespace. You can use the template in the ~/DO280/labs/operators-cli/operator-group.yaml path. Edit the file and configure the namespaces.

apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: file-integrity-operator
  namespace: openshift-file-integrity
spec:
  targetNamespaces:
  - openshift-file-integrity

Create the operator group.

[student@workstation ~]$ oc create \
  -f ~/DO280/labs/operators-cli/operator-group.yaml
operatorgroup.operators.coreos.com/file-integrity-operator created

Create the subscription in the operator namespace. You can use the template in the ~/DO280/labs/operators-cli/subscription.yaml path. Edit the file with the data that you obtained in a previous step. Set the approval policy to Manual.

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: file-integrity-operator
  namespace: openshift-file-integrity
spec:
  channel: "stable"
  installPlanApproval: Manual
  name: file-integrity-operator
  source: do280-catalog-cs
  sourceNamespace: openshift-marketplace

Create the subscription.

[student@workstation ~]$ oc create -f ~/DO280/labs/operators-cli/subscription.yaml
subscription.operators.coreos.com/file-integrity-operator created

Approve the install plan.

Examine the operator resource that the OLM created.

[student@workstation ~]$ oc describe operator file-integrity-operator
Name:         file-integrity-operator.openshift-file-integrity
...output omitted...
Status:
  Components:
    Label Selector:
      Match Expressions:
        Key:       operators.coreos.com/file-integrity-operator.openshift-file-integrity
        Operator:  Exists
    Refs:
...output omitted...
      Kind:                    InstallPlan
      Name:                    install-4wsq6
      Namespace:               openshift-file-integrity
      API Version:             operators.coreos.com/v1alpha1
      Conditions:
        Last Transition Time:  2024-01-26T10:38:22Z
        Message:               all available catalogsources are healthy
        Reason:                AllCatalogSourcesHealthy
        Status:                False
        Type:                  CatalogSourcesUnhealthy
        Last Transition Time:  2024-01-26T10:38:21Z
        Reason:                RequiresApproval
        Status:                True
        Type:                  InstallPlanPending
      Kind:                    Subscription
      Name:                    file-integrity-operator
      Namespace:               openshift-file-integrity
Events:                        <none>

Verify that the operator has a condition of the InstallPlanPending type. The operator can have other conditions, and they do not indicate a problem. The operator references the install plan. You use the install plan name in a later step. If the install plan is not generated, then wait a few moments and run the oc describe command again.

View the install plan specification with the oc get command. Replace the name with the install plan name that you obtained in a previous step.

[student@workstation ~]$ oc get installplan -n openshift-file-integrity \
  install-4wsq6 -o jsonpath='{.spec}{"\n"}'
{"approval":"Manual","approved":false,"clusterServiceVersionNames":["file-integrity-operator.v1.3.3","file-integrity-operator.v1.3.3"],"generation":1}

The install plan is set to manual approval, and the approved field is set to false.

Approve the install plan with the oc patch command. Replace the name with the install plan name that you obtained in a previous step.

[student@workstation ~]$ oc patch installplan install-4wsq6 --type merge -p \
    '{"spec":{"approved":true}}' -n openshift-file-integrity
installplan.operators.coreos.com/install-4wsq6 patched

Verify that the operator installs successfully, by using the oc describe command. Check the latest transaction for the current status. The installation might not complete immediately. If the installation is not complete, then wait a few minutes and view the status again.

[student@workstation ~]$ oc describe operator file-integrity-operator
...output omitted...
Status:
  Components:
    Label Selector:
      Match Expressions:
        Key:       operators.coreos.com/file-integrity-operator.openshift-file-integrity
        Operator:  Exists
    Refs:
      ...output omitted...
      Conditions:
        Last Transition Time:  2024-01-26T18:21:03Z
        Last Update Time:      2024-01-26T18:21:03Z
        Message:               install strategy completed with no errors
        Reason:                InstallSucceeded
        Status:                True
        Type:                  Succeeded
      Kind:                    ClusterServiceVersion
      Name:                    file-integrity-operator.v1.0.0
      Namespace:               openshift-file-integrity
      ...output omitted...

Examine the workloads in the openshift-file-integrity namespace.

[student@workstation ~]$ oc get all -n openshift-file-integrity
Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
NAME                                           READY   STATUS    RESTARTS      AGE
pod/file-integrity-operator-6985588576-x2k49   1/1     Running   1 (50s ago)   56s

...output omitted...

NAME                                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/file-integrity-operator   1/1     1            1           56s

NAME                                                 DESIRED   CURRENT   READY   AGE
replicaset.apps/file-integrity-operator-6985588576   1         1         1       56s

The namespace has a ready deployment.

Test the operator to ensure that it is functional. The operator watches FileIntegrity resources, runs file integrity checks on nodes, and creates FileIntegrityNodeStatus with the results of the checks.

Create a FileIntegrity custom resource by applying the file at ~/DO280/labs/operators-cli/worker-fileintegrity.yaml with the oc apply command.

[student@workstation ~]$ oc apply -f \
    ~/DO280/labs/operators-cli/worker-fileintegrity.yaml
fileintegrity.fileintegrity.openshift.io/worker-fileintegrity created

Verify that the operator functions, by viewing the worker-fileintegrity object with the oc describe command.

[student@workstation ~]$ oc describe fileintegrity worker-fileintegrity \
    -n openshift-file-integrity
Name:         worker-fileintegrity
Namespace:    openshift-file-integrity
Labels:       <none>
Annotations:  <none>
API Version:  fileintegrity.openshift.io/v1alpha1
Kind:         FileIntegrity
...output omitted...
Spec:
  Config:
    Grace Period:  900
    Max Backups:   5
  Node Selector:
    node-role.kubernetes.io/worker:
  Tolerations:
    Effect:    NoSchedule
    Key:       node-role.kubernetes.io/master
    Operator:  Exists
    Effect:    NoSchedule
    Key:       node-role.kubernetes.io/infra
    Operator:  Exists
Events:        <none>

Use oc edit to edit the Grace Period to 60 in the FileIntegrity custom resource to trigger a failure.

[student@workstation ~]$ oc edit fileintegrity worker-fileintegrity \
    -n openshift-file-integrity
Name:         worker-fileintegrity
Namespace:    openshift-file-integrity
Labels:       <none>
Annotations:  <none>
API Version:  fileintegrity.openshift.io/v1alpha1
Kind:         FileIntegrity
...output omitted...
Spec:
  Config:
    Grace Period:  60
    Max Backups:   5
  Node Selector:
    node-role.kubernetes.io/worker:
  Tolerations:
    Effect:    NoSchedule
    Key:       node-role.kubernetes.io/master
    Operator:  Exists
    Effect:    NoSchedule
    Key:       node-role.kubernetes.io/infra
    Operator:  Exists
Events:        <none>

Verify that the operator automatically creates a FileIntegrityNodeStatus object, by using the oc get command. You might need to wait a few minutes for the object to generate.
Note
The first file integrity resource that you create might not work correctly.
If the operator does not create the FileIntegrityNodeStatus resource in a few minutes, then delete the FileIntegrity resource and create it again.
The exercise outcome does not depend on obtaining a FileIntegrityNodeStatus resource.
```
[student@workstation ~]$ oc get fileintegritynodestatuses \
    -n openshift-file-integrity
NAME                            NODE       STATUS
worker-fileintegrity-master01   master01   Succeeded
```

After FileIntegrityNodeStatus has successfully been created, run this as the admin user to modify the node's filesystem: oc debug node/master01 — touch /host/etc/foobar

[student@workstation ~]$ oc debug node/master01 -- touch /host/etc/foobar
Starting pod/master01-debug-l92pd ...
To use host binaries, `run chroot /host`

Removing debug pod ...

Run oc get configmaps -n openshift-file-integrity to list configmaps in the openshift-file-integrity namespace.

[student@workstation ~]$ oc get configmaps -n openshift-file-integrity --watch
NAME                                          DATA    AGE
aide-pause                                    1       109m
aide-reinit                                   1       109m
aide-worker-fileintegrity-master01-failed     1       108m
kube-root-ca.crt                              1       117m
openshift-service-ca.crt                      1       117m
worker-fileintegrity                          1       109m

Note

It may take several minutes for aide-worker-fileintegrity-master01-failed to show. Use the --watch flag and wait a few minutes until the failed configmap shows to move on to the next step. Press Ctrl+C to exit.

Run oc describe to view the report in aide-worker-fileintegrity-master01-failed configmap in the openshift-file-integrity namespace.

[student@workstation ~]$ oc describe \
    configmap/aide-worker-fileintegrity-master01-failed \
    -n openshift-file-integrity
Name:         aide-worker-fileintegrity-master01-failed
Namespace:    openshift-file-integrity
Labels:       file-integrity.openshift.io/node=master01
              file-integrity.openshift.io/owner=worker-fileintegrity
              file-integrity.openshift.io/result-log=
Annotations:  file-integrity.openshift.io/files-added: 1
              file-integrity.openshift.io/files-changed: 0
              file-integrity.openshift.io/files-removed: 0

Data
\====
integritylog:
\----
Start timestamp: 2024-01-26 18:31:16 +0000 (AIDE 0.16)
AIDE found differences between database and filesystem!!

Summary:
  Total number of entries:  32359
  Added entries:              1
  Removed entries:            0
  Changed entries:            0

---------------------------------------------------
Added entries:
---------------------------------------------------

f++++++++++++++++: /hostroot/etc/cni/multus/certs/multus-client-2024-01-26-15-14-01.pem
f++++++++++++++++: /hostroot/etc/foobar

---------------------------------------------------
The attributes of the (uncompressed) database(s):
---------------------------------------------------

/hostroot/etc/kubernetes/aide.db.gz
  MD5      : UswXQiVa/VpjlXF1rCP0vA==
  SHA1     : s6t06MCRrDgc4xOWnX6vk5rflGU=
  RMD160   : jvDdvAOC7/tI0TjDe7Kzmy5nUk8=
  TIGER    : TjW192YTQBmG4oGza7siI6CBRnztgrp6
  SHA256   : E8rWurdI9HgGP6402qWY+lDAaLoGiyNs
             PEka/siI1F0=
  SHA512   : JPDhgoEnNiTaDLqawkGtHplRW8f6zm3g
             jDB3E6X6XM4+13yhjwh/pokFAp5BhRSc
             0C4XXibXsS4OYxYiE5hBaw==


End timestamp: 2024-01-26 18:31:45 +0000 (run time: 0m 29s)


BinaryData
\====

Events:  <none>

Finish

On the workstation machine, use the lab command to complete this exercise. This step is important to ensure that resources from previous exercises do not impact upcoming exercises.

[student@workstation ~]$ lab finish operators-cli

Discuss Red Hat OpenShift Administration II: Configuring a Production Cluster

Go to community

Red Hat OpenShift Administration II: Operating a Production Kubernetes Cluster (DO280)

Haley_Ruccio

26 lip 2023

Configure, manage, and troubleshoot OpenShift clusters and containerized applicationsRed Hat OpenShift Administration II: Operating a Production Kubernetes Cluster (DO280) prepares OpenShift Cluster Administrators to perform daily administration tasks on clusters that host applications provided by internal teams and external vendors, enable self-service for cluster users with different roles, and deploy applications that require special permissions such as such as CI/CD tooling, performance monitoring, and security scanners. This course focuses on configuring multi-tenancy and security features of OpenShift as well as managing OpenShift add-ons based on operators.The skills you learn in this course can be applied using all versions of OpenShift, including Red Hat OpenShift on AWS (ROSA), Azure Red Hat OpenShift, and OpenShift Container Platform.This course is based on OpenShift Container Platform 4.12.5-10 Course TopicsDeploying packaged applications using manifests, templates, kustomize, and helm.Configuring authentication and authorization for users and applications.Protecting network traffic with network policies and exposing applications with proper network access.Deploying and managing applications using resources manifests.Enabling developer self-service of application projects.Managing OpenShift cluster updates and Kubernetes operator updates.Target AudienceSystem Administrators and Platform Operators interested in the ongoing management of OpenShift clusters, applications, users, and add-ons.Site Reliability Engineers interested in the ongoing maintenance and troubleshooting of Kubernetes clusters.System and Software Architects interested in understanding the security of an OpenShift cluster.Recommended TrainingTake our free assessment to gauge whether this offering is the best fit for your skills.Prerequisite: Red Hat OpenShift I: Containers & Kubernetes (DO180 v4.12), or equivalent skills deploying and managing Kubernetes applications using the OpenShift web console and command-line interfaces.Technology considerationsThis course requires internet access to access the cloud-based classroom environment that provides an OpenShift cluster and a remote administrator’s workstation.

476

About pod security standards and warnings

alexcorcoles

17 kwi 2023

(This material has been created with the inputs of instructors and other people. We are trying to get this material published as soon as possible to address some concerns with DO180 and DO280.)With the default configuration, any namespace that you create on OpenShift 4.12 has the following labels: pod-security.kubernetes.io/audit: restrictedpod-security.kubernetes.io/audit-version: v1.24pod-security.kubernetes.io/warn: restrictedpod-security.kubernetes.io/warn-version: v1.24 These labels activate auditing and warning of the restricted pod security standard. The Kubernetes documentation describes the restricted pod security standard. Pod security standards such as restricted are not directly related to security context constraints such as the restricted SCC.Kubernetes uses an admission controller to reject pods that do not comply with pod security standards. An admission controller is a Kubernetes component that inspects every Kubernetes resource before its creation, and acts before Kubernetes creates the resource. This action can include rejecting the creation of the resource, or modifying the resource.The pod security standards are policies that include properties that workloads must follow. For example, the restricted standard requires that containers set the securityContext.allowPrivilegeEscalation field to false.Pod security standards apply both to workloads, such as deployments and jobs, and to pods.When a pod is created, the admission controller inspects the container specifications and enumerates the fields that violate the pod security standard of the namespace. The admission controller inspects the pod-security labels in the namespace to decide which pod security standard to apply (restricted, baseline, or privileged), and which action to take on violations. Namespaces can audit, warn, or enforce pod security standard violations. When the pod-security.kubernetes.io/audit annotation is set to restricted, the admission controller prevents the creation of pods that violate the security standard. When the pod-security.kubernetes.io/audit and pod-security.kubernetes.io/warn annotations are defined in the project, the admission controller does not block the creation of pods, but registers audit events or warnings.The admission controller does not enforce pod security standards for workloads. If the namespace is configured to enforce a pod security standard, then you can create a violating workload. The admission controller enforces only pod security policies on pods.Cluster administrators can use pod security standards to ensure that pods and workloads in namespaces limit their privileges. By limiting privileges, cluster administrators can mitigate what malicious pods can do.The default configuration of user namespaces in OpenShift does not restrict any workload; it only notifies users when they create workloads that do not follow pod security standards that might be enforced in future OpenShift versions.Security Context ConstraintsOpenShift introduced security context constraints (SCC) before the introduction of pod security standards in Kubernetes. SCCs automatically make security modifications to pods when they are created. By default, the system:openshift:scc:restricted-v2 cluster role binding applies to all authenticated users, so regular user workloads use the restricted-v2 SCC. For example, when you execute the following command to create a workload without any security configuration, the SCC adds security configuration to the pods: [user@host ~]$ oc create deployment test --image registry.ocp4.example.com:8443/redhattraining/hello-world-nginx (You can add the --dry-run=client -o yaml options to view the deployment manifest. The deployment manifest does not contain the security properties, which are described later.)If you inspect the pods that the deployment created, the SCC makes the following changes: [user@host ~]$ oc get pod test-b8d5658b6-7bxfs -o yamlapiVersion: v1kind: Podmetadata: annotations:... output omitted... openshift.io/scc: restricted-v2 seccomp.security.alpha.kubernetes.io/pod: runtime/default...output omitted...spec: containers: - image: registry.ocp4.example.com:8443/redhattraining/hello-world-nginx imagePullPolicy: Always name: hello-world-nginx resources: {} securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL runAsNonRoot: true runAsUser: 1000690000... output omitted... securityContext: fsGroup: 1000690000 seLinuxOptions: level: s0:c26,c20 seccompProfile: type: RuntimeDefault... output omitted... These restrictions satisfy the restricted pod security policy.The result of these defaults means that when creating a workload (such as a deployment) in a namespace without any customization, the restricted pod security standard applies, but the admission controller only warns and audits. If the workload does not satisfy the pod security standard, then the admission controller allows the workload to be created, but prints a message such as the following warning: Warning: would violate PodSecurity "restricted:v1.24": allowPrivilegeEscalation != false (container "example" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "example" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "database" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "example" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost") However, by default Kubernetes creates the deployment pods with the restricted-v2 SCC, with the restrictions that the previous warning included. Due to the following factors, workloads generate only a warning, and the pod security standard admission controller allows their pods:The pod security standard admission controller never enforces pod security standards on workloads (only on pods).By default, namespaces do not enforce pod security standards on pods.By default, regular workloads create pods by using the restricted-v2 SCC, which satisfies the restricted pod security standard.Even if future versions of OpenShift change the default namespace configuration to enforce the restricted pod security standard, users will continue to be able to create workloads without explicitly declaring the restrictions that the pod security standard requires, because the restricted-v2 SCC will apply such restrictions automatically.You can take the following approaches to handling pod security:For most workloads, the restricted-v2 SCC limits pod privileges adequately. When creating a workload, you receive the warning, but the workload creates pods and works correctly.You can also create your workloads by providing a full specification of the security constraints to apply, without relying on the SCC to provide the configuration. By following this approach, privileges are more explicit, and you have greater control, so you can adjust more precisely the specific privileges that are granted to workloads.When the restricted-v2 SCC and the default restricted pod security standard are not adequate for your workload, because either you require further privileges, or you need further restrictions, you must use an existing SCC, or create a custom SCC, and provide explicit security specifications. You can also adjust the namespace labels to choose a different pod security standard, and decide to audit, log, or enforce the standard.

1473

Welcome to the Red Hat OpenShift Administration II (DO280) group in the Red Hat Learning Community!

Deanna

17 kwi 2023

We are excited to launch a space dedicated to the Red Hat Training course Red Hat OpenShift Administration II: Operating a Production Kubernetes Cluster!To gain the most value from this group - click the "Join Group" button in the upper right hand corner.We encourage group members to collaborate in this group to discuss topics, ask questions, share best practices and tips, provide course feedback, and share their accomplishments as it relates to DO280.Read more about Red Hat OpenShift Administration II here.Thanks!

1334

Revision: do280-4.14-08d11e1