DO280 - ch06s05

Bookmark this page

The Project Template and the Self-Provisioner Role

Objectives

Configure default quotas, limit ranges, role bindings, and other restrictions for new projects, and the allowed users to self-provision new projects.

Project Creation

Kubernetes provides namespaces to isolate workloads.

Namespace metadata has security implications in clusters. For example, policy controllers might use namespace labels to limit capabilities in a namespace. If users can modify namespaces, then malicious users can modify namespace metadata to override security measures.

Additionally, namespaces are not namespaced. Therefore, granting granular access to namespaces poses some challenges. For example, with Kubernetes role-based access control, you cannot allow users to list a subset of namespaces. However, to allow users to list their namespaces, you must allow them to list all namespaces.

Note

Listing resources and viewing individual resources are different operations. You can grant users permissions to view specific namespaces, but listing namespaces requires a separate permission.

OpenShift introduces projects to improve security and users' experience of working with namespaces. The OpenShift API server adds the Project resource type. When you make a query to list projects, the API server lists namespaces, filters the visible namespaces to your user, and returns the visible namespaces in project format.

Additionally, OpenShift introduces the ProjectRequest resource type. When you create a project request, the OpenShift API server creates a namespace from a template. By using a template, cluster administrators can customize namespace creation. For example, cluster administrators can ensure that new namespaces have specific permissions, resource quotas, or limit ranges.

These features provide self-service management of namespaces. Cluster administrators can allow users to create namespaces without allowing users to modify namespace metadata. Administrators can also customize the creation of namespaces to ensure that namespaces follow organizational requirements.

Planning a Project Template

You can add any namespaced resource to the project template. For example, you can add resources of the following types:

Roles and role bindings

Add roles and role bindings to the template to grant specific permissions in new projects. The default template grants the admin role to the user who requests the project. You can keep this permission or use another similar permission, such as granting the admin role to a group of users. You can also add different permissions, such as more granular permissions over specific resource types.

Resource quotas and limit ranges

Add resource quotas to the project template to ensure that all new projects have resource limits. If you add resource quotas, then creating workloads requires explicit resource limit declarations. Consider adding limit ranges to reduce the effort for workload creation.

Even with quotas in all namespaces, users can create projects to continue adding workloads to a cluster. If this scenario is a concern, then consider adding cluster resource quotas to the cluster.

Network policies

Add network policies to the template to enforce organizational network isolation requirements.

Creating a Project Template

The oc adm create-bootstrap-project-template command prints a template that you can use to create your own project template.

This template has the same behavior as the default project creation in OpenShift. The template adds a role binding that grants the admin cluster role over the new namespace to the user who requests the project.

Project templates use the same template feature as the oc new-app command.

Execute the following command to create a file with an initial template:

[user@host ~]$ oc adm create-bootstrap-project-template -o yaml > file

This initial template has the following content:

apiVersion: template.openshift.io/v1
kind: Template
metadata:
  creationTimestamp: null
  name: project-request
objects: 
- apiVersion: project.openshift.io/v1 
  kind: Project
  metadata:
    annotations:
      openshift.io/description: ${PROJECT_DESCRIPTION}
      openshift.io/display-name: ${PROJECT_DISPLAYNAME}
      openshift.io/requester: ${PROJECT_REQUESTING_USER}
    creationTimestamp: null
    name: ${PROJECT_NAME}
  spec: {}
  status: {}
- apiVersion: rbac.authorization.k8s.io/v1 
  kind: RoleBinding
  metadata:
    creationTimestamp: null
    name: admin
    namespace: ${PROJECT_NAME}
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: admin
  subjects:
  - apiGroup: rbac.authorization.k8s.io
    kind: User
    name: ${PROJECT_ADMIN_USER}
parameters: 
- name: PROJECT_NAME
- name: PROJECT_DISPLAYNAME
- name: PROJECT_DESCRIPTION
- name: PROJECT_ADMIN_USER
- name: PROJECT_REQUESTING_USER

	The resources that OpenShift creates in new namespaces
	The project resource
	A role binding to grant the `admin` role to the requesting user
	The parameters that are available to the template

When a user requests a project, OpenShift replaces the ${VARIABLE} syntax with the parameters of the project request, and creates the objects in the objects key.

Modify the object list to add the required resources for new namespaces.

The YAML output of oc commands that return lists of objects is formatted similarly to the template objects key.

[user@host ~]$ oc get limitrange,resourcequota -o yaml
apiVersion: v1
items:
- apiVersion: v1
  kind: LimitRange
  metadata:
    creationTimestamp: "2024-01-31T17:48:23Z"
    name: example
    namespace: example
    resourceVersion: "881771"
    uid: d0c19c60-00a9-4028-acc5-22680f1ea658
  spec:
    limits:
    - default:
        cpu: 500m
        memory: 512Mi
      defaultRequest:
        cpu: 250m
        memory: 256Mi
      max:
        cpu: "1"
        memory: 1Gi
      min:
        cpu: 125m
        memory: 128Mi
      type: Container
- apiVersion: v1
  kind: ResourceQuota
  metadata:
    creationTimestamp: "2024-01-31T17:48:04Z"
    name: example
    namespace: example
    resourceVersion: "881648"
    uid: 108f0771-dc11-4289-ae76-6514d58bbece
  spec:
    hard:
      count/pods: "1"
  status:
...output omitted...
kind: List
metadata:
  resourceVersion: ""

Some common resources in project templates, such as quotas, do not have strict validation. For example, if the previous template contains the count/pod text instead of the count/pods text, then the quota does not work. You can create the project template, and new namespaces contain the quota, but the quota does not have an effect. To define a project template and to reduce the risk of errors, you can perform the following steps:

Create a namespace.
Create your chosen resources and test until you get the intended behavior.
List the resources in YAML format.
Edit the resource listing to ensure that the definitions create the correct resources. For example, remove elements that do not apply to resource creation, such as the creationTimestamp or status keys.
Replace the namespace name with the ${PROJECT_NAME} value.
Add the list of resources to the project template that the oc adm create-bootstrap-project-template command generates.

Note

Extracting a resource definition from an existing resource might not always produce correct results. Besides including elements that do not apply to resource creation, existing definitions might contain attributes that generate unexpected behavior. For example, a controller might add to resources some annotations that are unsuitable for template definitions.

Even after testing the resources in a test namespace, always verify that the projects that are created from your template have only the required behavior.

Use the oc create command to create the template resource in the openshift-config namespace:

[user@host ~]$ oc create -f template -n openshift-config
template.template.openshift.io/project-request created

Configuring the Project Template

Update the projects.config.openshift.io/cluster resource to use the new project template. Modify the spec section. By default, the name of the project template is project-request.

apiVersion: config.openshift.io/v1
kind: Project
metadata:
...output omitted...
  name: cluster
...output omitted...
spec:
  projectRequestTemplate:
    name: project-request

A successful update to the projects.config.openshift.io/cluster resource rolls out a new version of the apiserver deployment in the openshift-apiserver namespace. After the new apiserver deployment completes, new projects create the resources in the customized project template.

Note

During the apiserver deployment rollout, API requests can produce unexpected results.

To revert to the original project template, modify the projects.config.openshift.io/cluster resource to clear the spec resource to match the spec: {} format.

Managing Self-provisioning Permissions

Users with the self-provisioner cluster role can create projects. By default, the self-provisioner role is bound to all authenticated users.

Control the binding of the role to limit which users can request new projects.

Important

Remember that users with namespace permissions can create namespaces that do not use the project template.

Use the oc describe command to view the role bindings.

[user@host ~]$ oc describe clusterrolebinding.rbac self-provisioners
Name:         self-provisioners
Labels:       <none>
Annotations:  rbac.authorization.kubernetes.io/autoupdate: true
Role:
  Kind:  ClusterRole
  Name:  self-provisioner
Subjects:
  Kind   Name                        Namespace
  ----   ----                        ---------
  Group  system:authenticated:oauth

This role binding has an rbac.authorization.kubernetes.io/autoupdate annotation. This annotation protects roles and bindings from modifications that can interfere with the working of clusters. When the API server starts, the cluster restores resources with this annotation automatically, unless you set the annotation to the false value.

To make changes, disable automatic updates with the annotation, and edit the subjects in the binding.

Important

The oc adm policy remove-cluster-role-from-group command removes the cluster role binding when you remove the last subject.

Use extra caution or avoid this command to manage protected role bindings. The command removes the permission, but only until the API server restarts. Removing the permission permanently after deleting the binding is a lengthier process than changing the subjects.

For example, to disable self-provisioning, execute the following commands:

[user@host ~]$ oc annotate clusterrolebinding/self-provisioners \
  --overwrite rbac.authorization.kubernetes.io/autoupdate=false
clusterrolebinding.rbac.authorization.k8s.io/self-provisioners annotated
[user@host ~]$ oc patch clusterrolebinding.rbac self-provisioners \
  -p '{"subjects": null}'
clusterrolebinding.rbac.authorization.k8s.io/self-provisioners patched

You can also use the oc edit command to modify any value of a resource. The command launches the vi editor to apply your modifications. For example, to change the subject of the role binding from the system:authenticated:oauth group to the provisioners group, execute the followign command:

[user@host ~]$ oc edit clusterrolebinding/self-provisioners
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
...output omitted...
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: self-provisioner
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: provisioners

References

For more information, refer to the Configuring Project Creation section in the Projects chapter in the Red Hat OpenShift Container Platform 4.14 Building Applications documentation at https://access.redhat.com/documentation/en-us/openshift_container_platform/4.14/html-single/building_applications/index#configuring-project-creation

Customizing OpenShift Project Creation

Discuss Red Hat OpenShift Administration II: Configuring a Production Cluster

Go to community

Red Hat OpenShift Administration II: Operating a Production Kubernetes Cluster (DO280)

Haley_Ruccio

26 lip 2023

Configure, manage, and troubleshoot OpenShift clusters and containerized applicationsRed Hat OpenShift Administration II: Operating a Production Kubernetes Cluster (DO280) prepares OpenShift Cluster Administrators to perform daily administration tasks on clusters that host applications provided by internal teams and external vendors, enable self-service for cluster users with different roles, and deploy applications that require special permissions such as such as CI/CD tooling, performance monitoring, and security scanners. This course focuses on configuring multi-tenancy and security features of OpenShift as well as managing OpenShift add-ons based on operators.The skills you learn in this course can be applied using all versions of OpenShift, including Red Hat OpenShift on AWS (ROSA), Azure Red Hat OpenShift, and OpenShift Container Platform.This course is based on OpenShift Container Platform 4.12.5-10 Course TopicsDeploying packaged applications using manifests, templates, kustomize, and helm.Configuring authentication and authorization for users and applications.Protecting network traffic with network policies and exposing applications with proper network access.Deploying and managing applications using resources manifests.Enabling developer self-service of application projects.Managing OpenShift cluster updates and Kubernetes operator updates.Target AudienceSystem Administrators and Platform Operators interested in the ongoing management of OpenShift clusters, applications, users, and add-ons.Site Reliability Engineers interested in the ongoing maintenance and troubleshooting of Kubernetes clusters.System and Software Architects interested in understanding the security of an OpenShift cluster.Recommended TrainingTake our free assessment to gauge whether this offering is the best fit for your skills.Prerequisite: Red Hat OpenShift I: Containers & Kubernetes (DO180 v4.12), or equivalent skills deploying and managing Kubernetes applications using the OpenShift web console and command-line interfaces.Technology considerationsThis course requires internet access to access the cloud-based classroom environment that provides an OpenShift cluster and a remote administrator’s workstation.

476

About pod security standards and warnings

alexcorcoles

17 kwi 2023

(This material has been created with the inputs of instructors and other people. We are trying to get this material published as soon as possible to address some concerns with DO180 and DO280.)With the default configuration, any namespace that you create on OpenShift 4.12 has the following labels: pod-security.kubernetes.io/audit: restrictedpod-security.kubernetes.io/audit-version: v1.24pod-security.kubernetes.io/warn: restrictedpod-security.kubernetes.io/warn-version: v1.24 These labels activate auditing and warning of the restricted pod security standard. The Kubernetes documentation describes the restricted pod security standard. Pod security standards such as restricted are not directly related to security context constraints such as the restricted SCC.Kubernetes uses an admission controller to reject pods that do not comply with pod security standards. An admission controller is a Kubernetes component that inspects every Kubernetes resource before its creation, and acts before Kubernetes creates the resource. This action can include rejecting the creation of the resource, or modifying the resource.The pod security standards are policies that include properties that workloads must follow. For example, the restricted standard requires that containers set the securityContext.allowPrivilegeEscalation field to false.Pod security standards apply both to workloads, such as deployments and jobs, and to pods.When a pod is created, the admission controller inspects the container specifications and enumerates the fields that violate the pod security standard of the namespace. The admission controller inspects the pod-security labels in the namespace to decide which pod security standard to apply (restricted, baseline, or privileged), and which action to take on violations. Namespaces can audit, warn, or enforce pod security standard violations. When the pod-security.kubernetes.io/audit annotation is set to restricted, the admission controller prevents the creation of pods that violate the security standard. When the pod-security.kubernetes.io/audit and pod-security.kubernetes.io/warn annotations are defined in the project, the admission controller does not block the creation of pods, but registers audit events or warnings.The admission controller does not enforce pod security standards for workloads. If the namespace is configured to enforce a pod security standard, then you can create a violating workload. The admission controller enforces only pod security policies on pods.Cluster administrators can use pod security standards to ensure that pods and workloads in namespaces limit their privileges. By limiting privileges, cluster administrators can mitigate what malicious pods can do.The default configuration of user namespaces in OpenShift does not restrict any workload; it only notifies users when they create workloads that do not follow pod security standards that might be enforced in future OpenShift versions.Security Context ConstraintsOpenShift introduced security context constraints (SCC) before the introduction of pod security standards in Kubernetes. SCCs automatically make security modifications to pods when they are created. By default, the system:openshift:scc:restricted-v2 cluster role binding applies to all authenticated users, so regular user workloads use the restricted-v2 SCC. For example, when you execute the following command to create a workload without any security configuration, the SCC adds security configuration to the pods: [user@host ~]$ oc create deployment test --image registry.ocp4.example.com:8443/redhattraining/hello-world-nginx (You can add the --dry-run=client -o yaml options to view the deployment manifest. The deployment manifest does not contain the security properties, which are described later.)If you inspect the pods that the deployment created, the SCC makes the following changes: [user@host ~]$ oc get pod test-b8d5658b6-7bxfs -o yamlapiVersion: v1kind: Podmetadata: annotations:... output omitted... openshift.io/scc: restricted-v2 seccomp.security.alpha.kubernetes.io/pod: runtime/default...output omitted...spec: containers: - image: registry.ocp4.example.com:8443/redhattraining/hello-world-nginx imagePullPolicy: Always name: hello-world-nginx resources: {} securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL runAsNonRoot: true runAsUser: 1000690000... output omitted... securityContext: fsGroup: 1000690000 seLinuxOptions: level: s0:c26,c20 seccompProfile: type: RuntimeDefault... output omitted... These restrictions satisfy the restricted pod security policy.The result of these defaults means that when creating a workload (such as a deployment) in a namespace without any customization, the restricted pod security standard applies, but the admission controller only warns and audits. If the workload does not satisfy the pod security standard, then the admission controller allows the workload to be created, but prints a message such as the following warning: Warning: would violate PodSecurity "restricted:v1.24": allowPrivilegeEscalation != false (container "example" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "example" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "database" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "example" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost") However, by default Kubernetes creates the deployment pods with the restricted-v2 SCC, with the restrictions that the previous warning included. Due to the following factors, workloads generate only a warning, and the pod security standard admission controller allows their pods:The pod security standard admission controller never enforces pod security standards on workloads (only on pods).By default, namespaces do not enforce pod security standards on pods.By default, regular workloads create pods by using the restricted-v2 SCC, which satisfies the restricted pod security standard.Even if future versions of OpenShift change the default namespace configuration to enforce the restricted pod security standard, users will continue to be able to create workloads without explicitly declaring the restrictions that the pod security standard requires, because the restricted-v2 SCC will apply such restrictions automatically.You can take the following approaches to handling pod security:For most workloads, the restricted-v2 SCC limits pod privileges adequately. When creating a workload, you receive the warning, but the workload creates pods and works correctly.You can also create your workloads by providing a full specification of the security constraints to apply, without relying on the SCC to provide the configuration. By following this approach, privileges are more explicit, and you have greater control, so you can adjust more precisely the specific privileges that are granted to workloads.When the restricted-v2 SCC and the default restricted pod security standard are not adequate for your workload, because either you require further privileges, or you need further restrictions, you must use an existing SCC, or create a custom SCC, and provide explicit security specifications. You can also adjust the namespace labels to choose a different pod security standard, and decide to audit, log, or enforce the standard.

1473

Welcome to the Red Hat OpenShift Administration II (DO280) group in the Red Hat Learning Community!

Deanna

17 kwi 2023

We are excited to launch a space dedicated to the Red Hat Training course Red Hat OpenShift Administration II: Operating a Production Kubernetes Cluster!To gain the most value from this group - click the "Join Group" button in the upper right hand corner.We encourage group members to collaborate in this group to discuss topics, ask questions, share best practices and tips, provide course feedback, and share their accomplishments as it relates to DO280.Read more about Red Hat OpenShift Administration II here.Thanks!

1334

Revision: do280-4.14-08d11e1