Bookmark this page

Lab: Cluster Self-service Setup

Configure a cluster with default settings for self-service projects.

Outcomes

  • Create a project template that sets quotas, ranges, and network policies.

  • Restrict access to the self-provisioners cluster role.

  • Create groups and assign users to groups.

  • Use role-based access control (RBAC) to grant permissions to groups.

As the student user on the workstation machine, use the lab command to prepare your system for this exercise.

[student@workstation ~]$ lab start compreview-review

The lab command copies the exercise files to the ~/DO280 directory and creates the following users:

  • do280-support

  • do280-platform

  • do280-presenter

  • do280-attendee

The goal, as the cluster administrator, is to configure a dedicated cluster to host workshops on different topics.

Each workshop requires a project, so that workshops are isolated from each other.

You must set up the cluster so that when the presenter creates a workshop project, the project gets a base configuration.

The presenter must be mostly self-sufficient to administer a workshop with little help from the workshop support team.

The workshop support team must deploy applications that administer workshops and that enhance the workshop experience. You set up a project and the applications for this purpose on a second lab.

Specifications

Use the following values to access the OpenShift cluster:

ItemValue
Dev user/password developer/developer
Admin user/password admin/redhatocp
API URL https://api.ocp4.example.com:6443

The following workshop groups are required:

  • Create the groups with the specified users in the following table:

    GroupUser
    platform do280-platform
    presenters do280-presenter
    workshop-support do280-support

    The lab start command creates the users with the redhat password.

  • The platform group administers the cluster.

  • The presenters group consists of the people who deliver the workshops.

  • The workshop-support group maintains the needed applications to support the workshops and the workshop presenters.

  • Ensure that only users from the following groups can create projects:

    Group
    platform
    presenters
    workshop-support
  • An attendee must not be able to create projects. Because this exercise requires steps that restart the Kubernetes API server, this configuration must persist across API server restarts.

  • The workshop-support group requires the following roles in the cluster:

    • The admin role to administer projects

    • A custom role that is provided in the groups-role.yaml file You must create this custom role to enable support members to create workshop groups and to add workshop attendees.

  • The platform group must be able to administer the cluster without restrictions.

  • The workshop-support group must perform the following tasks for the workshop project:

    • Create a workshop-specific attendees group.

    • Assign the edit role to the attendees group.

    • Add users to the attendees group.

  • Each workshop must be hosted in an independent project.

  • All the resources that the cluster creates with a new workshop project must use workshop as the name for grading purposes.

  • Each workshop must enforce the following maximum constraints:

    • The project uses up to 2 CPUs.

    • The project uses up to 1 Gi of RAM.

    • The project requests up to 1.5 CPUs.

    • The project requests up to 750 Mi of RAM.

  • Each workshop must enforce constraints to prevent an attendee's workload from consuming all the allocated resources for the workshop:

    • A workload uses up to 750m CPUs.

    • A workload uses up to 750 Mi.

  • Each workshop must have a resource specification for workloads:

    • A default limit of 500m CPUs.

    • A default limit of 500 Mi of RAM.

    • A default request of 0.1 CPUs.

    • A default request of 250 Mi of RAM.

      You can use the templates that are provided in the quota.yaml, limitrange.yaml, and networkpolicy.yaml files.

  • Each workshop project must have this additional default configuration:

    • A local binding for the presenter user to the admin cluster role with the workshop name

    • The workshop=project_name label to help to identify the workshop workload

    • Must accept traffic only from within the same workshop or from the ingress controller.

  • Use the registry.ocp4.example.com:8443/redhattraining/hello-world-nginx:v1.0 image, which listens on the 8080 port, to simulate a workshop workload.

  • As the do280-presenter user, you must create a workshop with the do280 name.

  • As the do280-support user, you must create the do280-attendees group with the do280-attendee user, and assign the edit cluster role to the do280-attendees group.

  1. Change to the ~/DO280/labs/compreview-review directory and log in to the cluster as the admin user.

    1. Change to the lab directory.

      [student@workstation ~]$ cd ~/DO280/labs/compreview-review
    2. Open a terminal window and log in as the admin user with the redhatocp password.

      [student@workstation compreview-review]$ oc login -u admin -p redhatocp \
        https://api.ocp4.example.com:6443
      Login successful.
      ...output omitted...
  2. Create the following groups and add a user as specified in the following table.

    GroupUser
    workshop-support do280-support
    presenters do280-presenter
    platform do280-platform
    1. Create the workshop-support group.

      [student@workstation compreview-review]$ oc adm groups new workshop-support
      group.user.openshift.io/workshop-support created
    2. Add the do280-support user to the workshop-support group.

      [student@workstation compreview-review]$ oc adm groups add-users \
        workshop-support do280-support
      group.user.openshift.io/workshop-support added: "do280-support"
    3. Create the presenters group.

      [student@workstation compreview-review]$ oc adm groups new presenters
      group.user.openshift.io/presenters created
    4. Add the do280-presenter user to the presenters group.

      [student@workstation compreview-review]$ oc adm groups add-users \
        presenters do280-presenter
      group.user.openshift.io/presenters added: "do280-presenter"
    5. Create the platform group.

      [student@workstation compreview-review]$ oc adm groups new platform
      group.user.openshift.io/platform created
    6. Add the do280-platform user to the platform group.

      [student@workstation compreview-review]$ oc adm groups add-users \
        platform do280-platform
      group.user.openshift.io/platform added: "do280-platform"
    7. Use the oc get groups command to verify that the group configuration is correct.

      [student@workstation compreview-review]$ oc get groups
      NAME                               USERS
      ...output omitted...
      platform                           do280-platform
      presenters                         do280-presenter
      workshop-support                   do280-support
  3. Grant to the workshop-support group the admin and the custom manage-groups cluster roles. You must create the manage-groups custom cluster role from the groups⁠-⁠role⁠.⁠yaml file.

    1. Grant the admin cluster role to the workshop-support group.

      [student@workstation compreview-review]$ oc adm policy \
        add-cluster-role-to-group admin workshop-support
      clusterrole.rbac.authorization.k8s.io/admin added: "workshop-support"
    2. Run the oc create command to create the manage-groups cluster role in the groups-role.yaml file.

      [student@workstation compreview-review]$ oc create -f groups-role.yaml
      clusterrole.rbac.authorization.k8s.io/manage-groups created
    3. Grant the manage-groups cluster role to the workshop-support group.

      [student@workstation compreview-review]$ oc adm policy \
        add-cluster-role-to-group manage-groups workshop-support
      clusterrole.rbac.authorization.k8s.io/manage-groups added: "workshop-support"
  4. Create a cluster role binding to assign the cluster-admin cluster role to the platform group.

    [student@workstation compreview-review]$ oc adm policy \
      add-cluster-role-to-group cluster-admin platform
    clusterrole.rbac.authorization.k8s.io/cluster-admin added: "platform"
  5. Allow only the platform, workshop-support and presenters groups to create projects, by editing the self-provisioner cluster role. Enforce that only users from these groups can create projects. Also, make this change permanent by setting the rbac.authorization.kubernetes.io/autoupdate annotation with the false value.

    1. Use the oc edit command to edit the self-provisioners cluster role binding.

      [student@workstation compreview-review]$ oc edit clusterrolebinding \
        self-provisioners

      Replace the subject of the role binding for the system:authenticated:oauth group with the platform, workshop-support, and presenters groups.

      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRoleBinding
      metadata:
        annotations:
          rbac.authorization.kubernetes.io/autoupdate: "false"
        creationTimestamp: "2023-01-24T23:31:00Z"
        name: self-provisioners
        resourceVersion: "250330"
        uid: a6053896-f68f-41ff-9bb3-5da579a701bc
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: ClusterRole
        name: self-provisioner
      subjects:
      - apiGroup: rbac.authorization.k8s.io
        kind: Group
        name: platform
      - apiGroup: rbac.authorization.k8s.io
        kind: Group
        name: workshop-support
      - apiGroup: rbac.authorization.k8s.io
        kind: Group
        name: presenters
    2. As the do280-attendee user, verify that you cannot create a project.

      Log in as the do280-attendee user with the redhat password.

      [student@workstation compreview-review]$ oc login -u do280-attendee -p redhat
      Login successful.
      
      You don't have any projects. Contact your system administrator to request a project.

      Use the oc new-project command to try to create a template-test project.

      [student@workstation compreview-review]$ oc new-project template-test
      Error from server (Forbidden): You may not request a new project via this API.
  6. As the admin user, create a template-test namespace to design the project template.

    1. Log in as the admin user with the redhatocp password.

      [student@workstation compreview-review]$ oc login -u admin -p redhatocp
      Login successful.
      ...output omitted...
    2. Use the oc new-project command to create the template-test project.

      [student@workstation compreview-review]$ oc new-project template-test
      Now using project "template-test" on server...
      ...output omitted...
  7. Create a template resource quota with the following specification.

    QuotaValue
    limits.cpu 2
    limits.memory 1Gi
    requests.cpu 1500m
    requests.memory 750Mi
    1. Edit the quota.yaml file and replace the CHANGE_ME label to match the following definition.

      apiVersion: v1
      kind: ResourceQuota
      metadata:
       name: workshop
       namespace: template-test
      spec:
        hard:
          limits.cpu: 2
          limits.memory: 1Gi
          requests.cpu: 1500m
          requests.memory: 750Mi
    2. Use the oc create command to create the quota in the template-test project.

      [student@workstation compreview-review]$ oc create -f quota.yaml
      resourcequota/workshop created
  8. Create the workshop limit range with the following specification.

    Limit typeValue
    max.cpu 750m
    max.mem 750Mi
    default.cpu 500m
    default.memory 500Mi
    defaulRequest.cpu 100m
    defaulRequest.memory 250Mi
    1. Edit the limitrange.yaml file and replace the CHANGE_ME label to match the following definition.

      apiVersion: v1
      kind: LimitRange
      metadata:
       name: workshop
       namespace: template-test
      spec:
       limits:
         - max:
             cpu: 750m
             memory: 750Mi
           default:
             cpu: 500m
             memory: 500Mi
           defaultRequest:
             cpu: 100m
             memory: 250Mi
           type: Container
    2. Use the oc create command to create the limit range in the template-test project.

      [student@workstation compreview-review]$ oc create -f limitrange.yaml
      limitrange/workshop created
  9. Create a network policy to accept traffic from within the workshop project or from outside the cluster. To identify the workshop project traffic, label the template-test namespace with the workshop=template-test label.

    1. Use the oc create deployment command to create a deployment without resource specifications.

      [student@workstation compreview-review]$ oc create deployment test-workload \
        --image registry.ocp4.example.com:8443/redhattraining/hello-world-nginx:v1.0
      deployment.apps/test-workload created
    2. Get the IP address of one of the NGINX pods.

      [student@workstation compreview-review]$ oc get pod -o wide
      NAME                                 READY   STATUS    ...   IP     ...
      test-workload-56bf7dc6fc-mshn9   1/1     Running   ... 10.8.0.138  ...
    3. Use the oc debug command to run the curl command from a pod in the default project.

      Use the curl command from the default namespace to query the NGINX server that runs in the test workload.

      [student@workstation compreview-review]$ oc debug --to-namespace="default" \
        -- curl -s http://10.8.0.138:8080
      Starting pod/image-debug ...
      <html>
        <body>
          <h1>Hello, world from nginx!</h1>
        </body>
      </html>
      
      Removing debug pod ...
    4. Use the oc label command to add the label to the template-test namespace.

      [student@workstation compreview-review]$ oc label ns template-test \
        workshop=template-test
      namespace/template-test labeled
    5. Edit the network policy from the networkpolicy.yaml file. Replace the CHANGE_ME labels according to the following specification.

      kind: NetworkPolicy
      apiVersion: networking.k8s.io/v1
      metadata:
        name: workshop
        namespace: template-test
      spec:
        podSelector: {}
        ingress:
          - from:
            - namespaceSelector:
                matchLabels:
                  workshop: template-test
            - namespaceSelector:
                matchLabels:
                  policy-group.network.openshift.io/ingress: ""
    6. Run the oc create command to create the policy in the template-test project.

      [student@workstation compreview-review]$ oc create -f networkpolicy.yaml
      networkpolicy.networking.k8s.io/workshop created
    7. Verify that you cannot connect to the workshop pod from the default project.

      [student@workstation compreview-review]$ oc debug --to-namespace="default" \
        -- curl -sS --connect-timeout 5 http://10.8.0.138:8080
      Starting pod/image-debug ...
      curl: (28) Connection timed out after 5000 milliseconds
      
      Removing debug pod ...
    8. Verify that you can connect to the workshop pod from the workshop project.

      [student@workstation compreview-review]$ oc debug \
        --to-namespace="template-test" \
        -- curl -sS http://10.8.0.138:8080
      Warning: would violate PodSecurity "restricted:latest": ...output omitted...
      Starting pod/image-debug ...
      <html>
        <body>
          <h1>Hello, world from nginx!</h1>
        </body>
      </html>
      
      Removing debug pod ...
  10. Create the workshop project template by using the previously created template resources.

    1. Run the oc adm create-bootstrap-project-template command to create the project-template.yaml file to use as the template for new projects.

      [student@workstation compreview-review]$ oc adm \
        create-bootstrap-project-template \
        -o yaml > project-template.yaml
    2. Use the oc get command to create a YAML list with the following resources:

      • resourcequota/workshop

      • limitrange/workshop

      • networkpolicy/workshop

        Redirect the output to append to the project-template.yaml file.

        [student@workstation compreview-review]$ oc get resourcequota/workshop \
          limitrange/workshop \
          networkpolicy/workshop \
          -o yaml >> project-template.yaml
    3. Edit the project-template.yaml file to perform the following operations:

      • Cut the contents of the items stanza and paste them immediately before the parameters stanza. Keep the original indentation, because every YAML item of the list must appear at the beginning of the line.

      • Remove any left-over content after the parameters block.

      • Remove the following keys from the limit range and quota definitions:

        • creationTimestamp

        • resourceVersion

        • uid

        • status

        • generation

      • Replace the template-test text with the ${PROJECT_NAME} text.

      • Add the workshop=${PROJECT_NAME} label.

      • Rename the admin role binding with the workshop name.

      Use the search-and-replace editor function to replace the template-test string with the ${PROJECT_NAME} template parameter. Optionally, you can use the sed command if it is available.

      The solution file is in the ~/DO280/solutions/compreview-review/project-template.yaml path.

      [student@workstation compreview-review]$ sed -i \
        's/template-test/${PROJECT_NAME}/g' project-template.yaml

      Then, move the resource list to the objects key. The project-template.yaml file has the following expected content.

      apiVersion: template.openshift.io/v1
      kind: Template
      metadata:
        name: project-request
      objects:
      - apiVersion: project.openshift.io/v1
        kind: Project
        metadata:
          annotations:
            openshift.io/description: ${PROJECT_DESCRIPTION}
            openshift.io/display-name: ${PROJECT_DISPLAYNAME}
            openshift.io/requester: ${PROJECT_REQUESTING_USER}
          name: ${PROJECT_NAME}
          labels:
            workshop: ${PROJECT_NAME}
        spec: {}
      - apiVersion: rbac.authorization.k8s.io/v1
        kind: RoleBinding
        metadata:
          name: workshop
          namespace: ${PROJECT_NAME}
        roleRef:
          apiGroup: rbac.authorization.k8s.io
          kind: ClusterRole
          name: admin
        subjects:
        - apiGroup: rbac.authorization.k8s.io
          kind: User
          name: ${PROJECT_ADMIN_USER}
      - apiVersion: v1
        kind: ResourceQuota
        metadata:
          annotations:
          name: workshop
          namespace: ${PROJECT_NAME}
        spec:
          hard:
            limits.cpu: "2"
            limits.memory: 1Gi
            requests.cpu: 1500m
            requests.memory: 750Mi
      - apiVersion: v1
        kind: LimitRange
        metadata:
          annotations:
          name: workshop
          namespace: ${PROJECT_NAME}
        spec:
          limits:
          - default:
              cpu: 500m
              memory: 500Mi
            defaultRequest:
              cpu: 100m
              memory: 250Mi
            max:
              cpu: 750m
              memory: 750Mi
            type: Container
      - apiVersion: networking.k8s.io/v1
        kind: NetworkPolicy
        metadata:
          annotations:
          name: workshop
          namespace: ${PROJECT_NAME}
        spec:
          ingress:
          - from:
            - namespaceSelector:
                matchLabels:
                  workshop: ${PROJECT_NAME}
            - namespaceSelector:
                matchLabels:
                  policy-group.network.openshift.io/ingress: ""
          podSelector: {}
          policyTypes:
          - Ingress
      parameters:
      - name: PROJECT_NAME
      - name: PROJECT_DISPLAYNAME
      - name: PROJECT_DESCRIPTION
      - name: PROJECT_ADMIN_USER
      - name: PROJECT_REQUESTING_USER
    4. Create the project template in the project-template.yaml file by using the oc create command in the openshift-config namespace.

      [student@workstation compreview-review]$ oc create -f project-template.yaml \
        -n openshift-config
      template.template.openshift.io/project-request created
    5. Use the oc edit command to change the cluster project configuration.

      [student@workstation compreview-review]$ oc edit \
        projects.config.openshift.io cluster

      Edit the resource to match the following content:

      apiVersion: config.openshift.io/v1
      kind: Project
      metadata:
      ...output omitted...
        name: cluster
      ...output omitted...
      spec:
        projectRequestTemplate:
          name: project-request

      To edit the file, you use the default vi editor.

    6. Use the watch command to view the API server pods.

      [student@workstation compreview-review]$ watch oc get \
        pod -n openshift-apiserver

      Wait until new pods are created. Press Ctrl+C to exit the watch command.

  11. As the do280-presenter, create the do280 workshop project.

    1. Log in as the do280-presenter user with the redhat password.

      [student@workstation compreview-review]$ oc login -u do280-presenter -p redhat
      Login successful.
      ...output omitted...
    2. Use the oc new-project command to create the do280 project.

      [student@workstation compreview-review]$ oc new-project do280
      Now using project "do280" on server ...
      ...output omitted...
    3. Verify that the oc new-project command creates the following resources from the template:

      • Quota

      • Limit range

      • Network policy

        [student@workstation compreview-review]$ oc get resourcequota/workshop \
          limitrange/workshop \
          networkpolicy/workshop
        NAME                     AGE   REQUEST                   LIMIT
        resourcequota/workshop   95s   requests.cpu: 0/1500m ... limits.cpu: 0/2 ...
        NAME                  CREATED AT
        limitrange/workshop   2023-03-03T10:37:28Z
        NAME                                       POD-SELECTOR   AGE
        networkpolicy.networking.k8s.io/workshop   <none>         95s
    4. Verify that the do280 project definition has the workshop=do280 label.

      [student@workstation compreview-review]$ oc get project do280 -o yaml
      apiVersion: project.openshift.io/v1
      kind: Project
      metadata:
      ...output omitted...
        labels:
          workshop: do280
      ...output omitted...
        name: do280
        resourceVersion: "1293438"
      ...output omitted...
  12. As the do280-support user, create the do280-attendees group. Then, assign the edit cluster role to the do280-attendees group, and add the do280-attendee user to the group.

    1. Log in as the do280-support user with the redhat password.

      [student@workstation compreview-review]$ oc login -u do280-support -p redhat
      Login successful.
      ...output omitted...
    2. Create the do280-attendees group.

      [student@workstation compreview-review]$ oc adm groups new do280-attendees
      group.user.openshift.io/do280-attendees created
    3. Assign the edit role to the do280-attendees group in the do280-workshop project.

      Add the edit cluster role to the do280-attendees group in the do280 project.

      [student@workstation compreview-review]$ oc adm policy \
        add-role-to-group edit do280-attendees -n do280
      clusterrole.rbac.authorization.k8s.io/edit added: "do280-attendees"
    4. As the do280-attendee user, verify that you cannot access the do280 project.

      Log in as the do280-attendee user with the redhat password.

      [student@workstation compreview-review]$ oc login -u do280-attendee -p redhat
      Login successful.
      You don't have any projects. ...
    5. As the do280-support user, add the do280-attendee user to the do280-attendees group.

      Log in as the do280-support user with the redhat password.

      [student@workstation compreview-review]$ oc login -u do280-support -p redhat
      Login successful.
      ...output omitted...

      Use the oc adm groups command to add the do280-attendee user to the workshop-do280-attendees group.

      [student@workstation compreview-review]$ oc adm groups add-users \
        do280-attendees do280-attendee
      group.user.openshift.io/do280-attendees added: "do280-attendee"
    6. As the do280-attendee user, verify that you can create workloads in the do280 project.

      Log in as the do280-attendee user with the redhat password.

      [student@workstation compreview-review]$ oc login -u do280-attendee -p redhat
      Login successful.
      You have one project on this server: "do280"
      Using project "do280".

      Use the oc create deployment command to create a deployment without resource specifications.

      [student@workstation compreview-review]$ oc create deployment \
        attendee-workload \
        --image registry.ocp4.example.com:8443/redhattraining/hello-world-nginx:v1.0
      deployment.apps/attendee-workload created
  13. Change to the home directory to prepare for the next exercise.

    [student@workstation appsec-review]$ cd

Evaluation

As the student user on the workstation machine, use the lab command to grade your work. Correct any reported failures and rerun the command until successful.

[student@workstation ~]$ lab grade compreview-review

Finish

As the student user on the workstation machine, use the lab command to complete this exercise. This step is important to ensure that resources from previous exercises do not impact upcoming exercises.

[student@workstation ~]$ lab finish compreview-review

Revision: do280-4.14-08d11e1