Configure and manage stateful applications deployed on Red Hat OpenShift.
Outcomes
Create and attach a persistent volume claim to a deployment.
Attach a configuration map as ephemeral storage to run a database initialization script.
Create a stateful set as an alternative way of running a stateful application.
As the student user on the workstation machine, use the lab command to prepare
your environment for this exercise, and to ensure that all required resources are available.
[student@workstation ~]$ lab start deployments-stateful
This script creates a new project called deployments-stateful containing a MySQL database server deployment.
Instructions
Log in to Red Hat OpenShift and use the deployments-stateful project.
Log in to OpenShift as the developer user.
[student@workstation ~]$ oc login -u developer -p developer \
https://api.ocp4.example.com:6443
Login successful.
...output omitted...Ensure that you use the deployments-stateful project.
[student@workstation ~]$ oc project deployments-stateful
Already on project "deployments-stateful" on server "https://api.ocp4.example.com:6443".Verify that the database server is running. Note that the server might take a minute to start.
[student@workstation ~]$ oc get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
mysql-db 1/1 1 1 1mAttach a persistent volume claim (PVC) called mysql-db-pvc to the deployment, mounted to the /var/lib/mysql path within the pod.
The configuration in this exercise demonstrates the basics of persistent storage and is not production ready.
In a production-ready database server deployment, each replica needs to handle sharding, failover, and other high-availability tasks. Simply increasing the number of replicas with this setup will not work.
Use the oc set volume command to create a PVC for the deployment.
[student@workstation ~]$ oc set volumes deploy/mysql-db \
--add --name nfs-volume-storage --type pvc \
--claim-mode rwo --claim-size 1Gi --mount-path /var/lib/mysql \
--claim-name mysql-db-pvc
deployment.apps/mysql-db volume updatedVerify that the PVC exists and has the correct attributes.
[student@workstation ~]$ oc get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
mysql-db-pvc Bound pvc-f8...cc 1Gi RWO nfs-storage 82sVerify that the PVC is attached to the database deployment.
[student@workstation ~]$ oc get deploy/mysql-db -o \
jsonpath='{.spec.template.spec.containers[0].volumeMounts}' | jq
[
{
"mountPath": "/var/lib/mysql",
"name": "nfs-volume-storage"
}
]The trailing jq tool invocation formats the JSON output of the preceding command to ease viewing.
Use a configuration map as an ephemeral volume to add initialization data to the database.
View the contents of the init-db.sql script that initializes the database.
[student@workstation ~]$cd ~/DO288/labs/deployments-stateful[student@workstation deployments-stateful]$cat init-db.sqlDROP TABLE IF EXISTS `Item`; CREATE TABLE `Item` (`id` BIGINT not null auto_increment primary key, `description` VARCHAR(100), `done` BIT); INSERT INTO `Item` (`id`,`description`,`done`) VALUES (1,'Pick up newspaper', 0); INSERT INTO `Item` (`id`,`description`,`done`) VALUES (2,'Buy groceries', 1);
Create a configuration map called init-db-cm that contains the SQL script.
[student@workstation deployments-stateful]$ oc create cm init-db-cm \
--from-file init-db.sql
configmap/init-db-cm createdAdd the configuration map as a volume called init-db-volume to the deployment.
Specify the volume type as configmap and set the /tmp/init-db directory as the mount path.
[student@workstation deployments-stateful]$ oc set volumes deploy/mysql-db \
--add --name init-db-volume \
--configmap-name init-db-cm --type configmap \
--mount-path /tmp/init-db
deployment.apps/mysql-db volume updatedVerify that the configuration map is attached to the database deployment. The list of volumes contains a new entry for the configuration map volume.
[student@workstation deployments-stateful]$oc get deploy/mysql-db -o \ jsonpath='{.spec.template.spec.containers[0].volumeMounts}' | jq[ { "mountPath": "/var/lib/mysql", "name": "nfs-volume-storage" },{ "mountPath": "/tmp/init-db", "name": "init-db-volume" }]
Run the mounted script to initialize the database and verify that the data exist.
Use the mysql client to execute the database script in the /tmp/init-db volume.
Ignore the warning message.
[student@workstation deployments-stateful]$ oc rsh deploy/mysql-db mysql \
-udbuser -pdbuser items -e "source /tmp/init-db/init-db.sql"
...output omitted...Verify that the table is populated.
[student@workstation deployments-stateful]$ oc rsh deploy/mysql-db \
mysql -udbuser -pdbuser items -e "select * from Item;"
...output omitted...
+----+-------------------+------------+
| id | description | done |
+----+-------------------+------------+
| 1 | Pick up newspaper | 0x00 |
| 2 | Buy groceries | 0x01 |
+----+-------------------+------------+Scale the deployment down and then back up to observe that the data persists even if the pod is destroyed.
Scale down the mysql-db deployment.
[student@workstation deployments-stateful]$ oc scale --replicas 0 deploy/mysql-db
deployment.apps/mysql-db scaledVerify that no database server pods are running.
[student@workstation deployments-stateful]$ oc get po
No resources found in deployments-stateful namespace.Verify that the PVC still exists.
[student@workstation deployments-stateful]$ oc get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS ...
mysql-db-pvc Bound pvc-f8...cc 1Gi RWO nfs-storage ...Scale up the mysql-db deployment.
[student@workstation deployments-stateful]$ oc scale --replicas 1 deploy/mysql-db
deployment.apps/mysql-db scaledConnect to the database pod and verify that the table data still exists.
[student@workstation deployments-stateful]$ oc rsh deploy/mysql-db \
mysql -udbuser -pdbuser items -e "select * from Item;"
...output omitted...
+----+-------------------+------------+
| id | description | done |
+----+-------------------+------------+
| 1 | Pick up newspaper | 0x00 |
| 2 | Buy groceries | 0x01 |
+----+-------------------+------------+Scale the deployment to see how all replicas use the same PVC, then delete the deployment and PVC.
Scale up the mysql-db deployment.
[student@workstation deployments-stateful]$ oc scale --replicas 3 deploy/mysql-db
deployment.apps/mysql-db scaledObserve that still only one PVC exists.
[student@workstation deployments-stateful oc get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS ...
mysql-db-pvc Bound pvc-f8...cc 1Gi RWO nfs-storage ...Delete the mysql-db deployment.
[student@workstation deployments-stateful]$ oc delete deploy/mysql-db
deployment.apps "mysql-db" deletedDelete the mysql-db-pvc PVC.
[student@workstation deployments-stateful]$ oc delete pvc/mysql-db-pvc
persistentvolumeclaim "mysql-db-pvc" deletedCreate a stateful set for the database server that mounts the configuration map and PVC.
Review the mysql-stateful-set.yaml file.
In particular, note the pieces that define how to create and mount the PVC and the configuration map.
The stateful set is dependent on the configuration map created in a preceding step.
[student@workstation deployments-stateful]$cat mysql-stateful-set.yamlapiVersion: apps/v1 kind: StatefulSet ...output omitted... spec: ...output omitted... template: ...output omitted... spec: containers: - image: registry.ocp4.example.com:8443/rhel8/mysql-80 ...output omitted...volumeMounts: - name: mysql-db-pvc mountPath: /var/lib/mysql subPath: mysql-db - name: init-db-volume mountPath: /tmp/init-db volumes: - name: init-db-volume configMap: name: init-db-cm volumeClaimTemplates: - metadata: name: mysql-db-pvc spec: accessModes: [ "ReadWriteOnce" ] storageClassName: nfs-storage resources: requests: storage: 1Gi
Apply the manifest to create the stateful set. Ignore the warning.
[student@workstation deployments-stateful]$ oc apply -f mysql-stateful-set.yaml
...output omitted...
statefulset.apps/mysql-db createdObserve that a database server pod is running.
[student@workstation deployments-stateful]$ oc get po
NAME READY STATUS RESTARTS AGE
mysql-db-0 1/1 Running 0 42sNote that the pod name includes a numeric suffix. Each additional replica for the stateful set increments sequentially.
Initialize the database, delete the pod, and observe that the pod is recreated with the persisted data intact.
Run the init-db.sql script inside the pod to initialize the database.
Ignore the warning message.
[student@workstation deployments-stateful]$ oc rsh mysql-db-0 mysql -udbuser \
-pdbuser items -e "source /tmp/init-db/init-db.sql"
...output omitted...Delete the singular database server pod.
[student@workstation deployments-stateful]$ oc delete po/mysql-db-0
pod "mysql-db-0" deletedObserve that OpenShift recreates the pod.
[student@workstation deployments-stateful]$ oc get po
NAME READY STATUS RESTARTS AGE
mysql-db-0 1/1 Running 0 22sConnect to the database pod and verify that the data still exists.
[student@workstation deployments-stateful]$ oc rsh mysql-db-0 mysql -udbuser \
-pdbuser items -e "select * from Item;"
...output omitted...
+----+-------------------+------------+
| id | description | done |
+----+-------------------+------------+
| 1 | Pick up newspaper | 0x00 |
| 2 | Buy groceries | 0x01 |
+----+-------------------+------------+Scale the stateful set to observe that each replica uses a new PVC. This is in contrast to the deployment that shared a single PVC amongst all replicas.
Update the stateful set to have three replicas.
[student@workstation deployments-stateful]$ oc scale --replicas 3 \
statefulset/mysql-db
statefulset.apps/mysql-db scaledWait for the pods to start. Notice that each new pod has an incremented numeric suffix.
[student@workstation deployments-stateful]$oc get podNAME READY STATUS RESTARTS AGE mysql-db-0 1/1 Running 0 3m32smysql-db-11/1Running0 78smysql-db-21/1Running0 74s
Observe that additional PVCs exist for each new pod.
[student@workstation deployments-stateful]$oc get pvcNAME STATUS VOLUME CAPACITY ACCESS MODES ... mysql-db-pvc-mysql-db-0 Bound pvc-e9...92 1Gi RWO ...mysql-db-pvc-mysql-db-1Bound pvc-08...f0 1Gi RWO ...mysql-db-pvc-mysql-db-2Bound pvc-0e...33 1Gi RWO ...