Red Hat OpenShift Developer II: Building and Deploying Cloud-native Applications

Instructions

1. Log in to Red Hat OpenShift, use the deployments-strategy project and change to the exercise directory.

   1. Log in to OpenShift as the developer user:

      [student@workstation ~]$ oc login -u developer -p developer \
      https://api.ocp4.example.com:6443
      Login successful.
      ...output omitted...

   2. Ensure that you use the deployments-strategy project:

      [student@workstation ~]$ oc project deployments-strategy
      Already on project "deployments-strategy" on server "https://api.ocp4.example.com:6443"

   3. Change to the exercise directory:

      [student@workstation ~]$ cd ~/DO288/labs/deployments-strategy/users-db

2. Create an OpenShift application by using the sources from the users-db application in the classroom Git repository and deploy it to the cluster.

   1. Use the oc new-app command with the -o yaml option to create a manifest with the following parameters:

      • Name: users-db

      • Variables:

        • MYSQL_USER: developer

        • MYSQL_PASSWORD: redhat

        • MYSQL_DATABASE: users

      • Repository URL: https://git.ocp4.example.com/developer/DO288-apps

      • Repository context: apps/deployments-strategy/users-db

      • Use the deploymentconfig resource

      Execute the following command:

      [student@workstation users-db]$ oc new-app --as-deployment-config \
      --name users-db \
      -e MYSQL_USER=developer -e MYSQL_PASSWORD=redhat -e MYSQL_DATABASE=users \
      https://git.ocp4.example.com/developer/DO288-apps \
      --context-dir=apps/deployments-strategy/users-db \
      -o yaml > application.yaml
      ...no output expected...

   2. Create the application in the cluster by running the oc apply command with the -f option to provide the application.yaml file.

      [student@workstation users-db]$ oc apply -f application.yaml
      imagestream.image.openshift.io/mysql-80 created
      imagestream.image.openshift.io/users-db created
      buildconfig.build.openshift.io/users-db created
      deploymentconfig.apps.openshift.io/users-db created
      service/users-db created

   3. Wait until the database pod has a Running status.

      [student@workstation users-db]$ oc get pods
      NAME                READY   STATUS      RESTARTS   AGE
      users-db-1-build    0/1     Completed   0          37s
      users-db-1-deploy   0/1     Completed   0          11s
      users-db-1-zp8xx    1/1     Running     0          10s

      OpenShift creates the users-db-1-build to build the container image. Then the users-db-1-deploy pod creates the users-db-1_zp8xx_ pod as the users-db deployment configuration specifies.

3. Change the deployment strategy to Recreate by editing the application.yaml manifest. Also, set a post lifecycle hook to update the database when the deployment is complete by executing the import.sh script.

   The script runs the users.sql script to initialize the database. To do this, the script tries to connect to the database at most HOOK_RETRIES times, and sleeps HOOK_SLEEP seconds between attempts.

   The script implements retries because the hook pod starts at the same time as the database pod. Consequently, the database pod must become ready before the pod hook can execute the SQL script.

   1. Verify that the default deployment strategy for the users-db database is Rolling.

      [student@workstation users-db]$ oc describe dc/users-db
      ...output omitted...
      Strategy:   Rolling
      ...output omitted...

   2. Read the import.sh script in the users-db directory to understand the usage of the HOOK_RETRIES variable.

   3. Inspect the users.sql copy in the users-db directory. This SQL script creates a table called users and inserts three rows of data.

      [student@workstation users-db]$ cat users.sql
      CREATE TABLE IF NOT EXISTS users (
          user_id int(10) unsigned NOT NULL AUTO_INCREMENT,
          name varchar(100) NOT NULL,
          email varchar(100)  NOT NULL,
          PRIMARY KEY (user_id)) ENGINE=InnoDB DEFAULT CHARSET=utf8;
      
      insert into users(name,email) values ('user1', 'user1@example.com');
      insert into users(name,email) values ('user2', 'user2@example.com');
      insert into users(name,email) values ('user3', 'user3@example.com');

   4. Edit the DeploymentConfig in the application.yaml manifest to set the Recreate strategy with a post lifecycle hook that runs the import script in the /post-deploy/import.sh container path. To do this, update the strategy section by replacing the resources: {} line with the following content:

      - apiVersion: apps.openshift.io/v1
        kind: DeploymentConfig
        metadata:
      ...output omitted...
          name: users-db
        spec:
          replicas: 1
          selector:
            deploymentconfig: users-db
          strategy:
            type: Recreate
            recreateParams:
              post:
                failurePolicy: Abort
                execNewPod:
                  containerName: users-db
                  command: ["/post-deploy/import.sh"]
      ...output omitted...

      Note that the containerName key has the users-db value, which matches the container name in the DeploymentConfig template.

      OpenShift must abort the deployment and rollback to the previous version if the post hook fails.

      You have the solution available in the ~/DO288/DO288-apps/solutions/deployments-strategy/application.yaml path.

   5. Run the oc apply command to update the deployment configuration.

      [student@workstation users-db]$ oc apply -f application.yaml
      imagestream.image.openshift.io/mysql-80 configured
      imagestream.image.openshift.io/users-db configured
      buildconfig.build.openshift.io/users-db configured
      deploymentconfig.apps.openshift.io/users-db configured
      service/users-db configured

4. Verify that the post lifecycle hook pod fails and that the deployment configuration falls back to the first deployment.

   [student@workstation users-db]$ watch -n 2 oc get pods
   users-db-1-build       0/1     Completed   0          14m
   users-db-1-deploy      0/1     Completed   0          14m
   users-db-1-hqzxq       1/1     Running     0          20s
   users-db-3-deploy      0/1     Error       0          30s
   users-db-3-hook-post   0/1     Error       0          27s

   When the hook and the deployment pods reach the error state, press Ctrl+C to exit the watch command.

   The users-db-1-hqzxq pod is a new pod. During this process, OpenShift terminated the original pod, attempted to create a pod by using the new deployment configuration, and then, because of the hook failure, created the users-db-1-hqzxq pod by using the original deployment configuration.

5. Troubleshoot and fix the post lifecycle hook.

   1. Display the logs from the failed pod. The logs show that the script returns an error status without trying to connect to the database.

      [student@workstation users-db]$ oc logs users-db-3-hook-post
      Trying 0 times, sleeping 2 sec between tries:
      Too many tries, giving up

      Notice that the script has a default value of 0 for the HOOK_RETRIES variable, causing it to never connect to the database.

   2. Increase the number of times the script attempts to connect to the database server. Set the HOOK_RETRIES environment variable in the deployment configuration to a value of 5.

      [student@workstation users-db]$ oc set env dc/users-db HOOK_RETRIES=5
      deploymentconfig.apps.openshift.io/users-db updated

      Note

      Because the deployment configuration has a ConfigChange trigger that automatically deploys changes to the DeploymentConfig resource, updating the HOOK_RETRIES value triggers a new pod.

   3. Wait until the new post lifecycle hook pod is in the Completed status and a new database pod is in the Running status.

      Note that the pods might show in a different order.

      [student@workstation users-db]$ watch -n 2 oc get pods
      NAME                READY     STATUS    RESTARTS   ...
      mysql-1-deploy      0/1     Completed   0          5m26s
      users-db-3-deploy      0/1     Error       0          3m30s
      users-db-3-hook-post   0/1     Error       0          3m8s
      users-db-4-29jwt       1/1     Running     0          57s
      users-db-4-deploy      0/1     Completed   0          79s
      users-db-4-hook-post   0/1     Completed   0          48s

      After the users-db-4-hook-post pod runs and exits, press Ctrl+C to exit the watch command.

      Notice that OpenShift retains the failed pods that were created during the previous deployment attempts, so you can display their logs for troubleshooting.

   4. Inspect the logs from the completed post lifecycle hook pod. They show that the script can connect to the database a few times, and then returns a success status.

      [student@workstation users-db]$ oc logs -f users-db-4-hook-post
      Trying 5 times, sleeping 2 sec between tries:
      Checking if MySQL is up...Database is up
      ...output omitted...
      Database initialized successfully

      The number of tries in the previous output might vary.

6. Verify that the new database pod contains the data from the SQL file.

   1. Use the oc get pods command to get the name of the current users-db pod.

      [student@workstation users-db]$ oc get pods -l deploymentconfig=users-db
      NAME            READY     STATUS    RESTARTS   AGE
      users-db-4-29jwt   1/1       Running   0          8m

   2. Open a shell session to the running users-db container pod.

      [student@workstation users-db]$ oc rsh users-db-4-29jwt \
       /bin/sh -c \
       'mysql -u$MYSQL_USER -p$MYSQL_PASSWORD $MYSQL_DATABASE -e "select * from users;"'
      ...output omitted...
      \+---------+-------\+-------------------+
      | user_id | name  | email             |
      \+---------+-------\+-------------------+
      |       1 | user1 | user1@example.com |
      |       2 | user2 | user2@example.com |
      |       3 | user3 | user3@example.com |
      \+---------+-------\+-------------------+