Red Hat OpenShift Developer II: Building and Deploying Cloud-native Applications

Stateful applications often require persisting data to permanent storage. This enables applications to recover from crashes and reboots and load the necessary state from storage. Some stateful applications, such as distributed stateful applications, might have additional networking requirements. For example, the Etcd distributed key-value Kubernetes store uses the Raft algorithm to maintain state between multiple Etcd replicas. This means that each replica pod must have a predictable network address to join the Etcd quorum.

Alternatively, a stateless application or service does not maintain any state which cannot be calculated at runtime. For example, an application that provides an API for calculating taxes from a set of formulas can respond directly from request parameters.

Generally speaking, stateless services are easier to maintain as they cannot run into state-based bugs and are simpler to scale. As demand on the stateless services changes, developers can add or remove replicas as necessary without dealing with data allocation or migration.

By isolating or delegating the stateful pieces, many large systems can consist of mostly stateless services. However, most applications need to store some state. For example, many applications delegate state persistence to a database server.

To address the storage needs of stateful applications and services, Kubernetes and Red Hat OpenShift provide the persistent volume subsystem.

Containers only provide ephemeral storage by default. When an application stores data to the container file system, the internal data is lost when the container is re-created.

To persist data, such as that used by a database server, Kubernetes and Red Hat OpenShift provide the persistent volume subsystem.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: data-volume 
spec:
  capacity:
    storage: 5Gi 

  volumeMode: Filesystem 
  accessModes:
    - ReadWriteOnce 
  persistentVolumeReclaimPolicy: Recycle 
  storageClassName: fast 
  nfs: 
    path: /exports-ocp4/example
    server: 192.168.50.10

apiVersion: v1
kind: PersistentVolumeClaim
  name: data-volume-claim 
spec:
  accessModes:
  - ReadWriteOnce 
  resources:
    requests:
      storage: 1Gi 
  volumeMode: Filesystem 
  ...output omitted...

The preceding examples assume a static provisioning technique. In other words, administrators manually create and maintain the PVs.

More commonly, teams are adopting a dynamic provisioning practice. Dynamic provisioning is when the cluster creates a new PV specifically for a PVC. Developers create a PVC and the cluster dynamically creates a PV for the PVC.

To use dynamic provisioning, the administrator must enable it for the cluster and create storage classes. Additionally, if a default storage class is not defined, then the PVC must specify one to be dynamically provisioned.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
  name: nfs-storage
parameters:
  archiveOnDelete: "false"
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner
reclaimPolicy: Delete
volumeBindingMode: Immediate

apiVersion: v1
kind: PersistentVolumeClaim
  name: dynamic-volume-claim
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: nfs-storage
  volumeMode: Filesystem
  ...output omitted...

You can mount a PVC that is bound to a PV to a pod as a volume.

For example, the following definition of a pod mounts a PVC called data-volume-claim inside of a pod called app-pod:

apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
    - name: app-ui
      image: quay.io/example/nginx
      volumeMounts:
      - mountPath: "/var/www/html" 
        name: ui-volume 
  volumes: 
    - name: ui-volume
      persistentVolumeClaim:
        claimName: data-volume-claim 

Note that the pod cannot start until all attached PVCs are bound.

Like bare pods, deployments can include definitions to mount PVCs within the resulting pods. Often, such a PVC is shared amongst all the replica pods. This is because each pod is identical to all other replica pods that the deployment manages.

Because most software is not built with shared storage in mind, deployments are often left to stateless applications. In particular, even though database servers are built with multi-pod and replication in mind, each database server instance often requires its own storage. Replication and sharding are often handled by the application.

The following is an example command to create and attach a PVC to an existing deployment called my-deployment:

[user@host ~]$ oc set volumes deploy/my-deployment \
--add \ 
--name nfs-volume-storage \
--type pvc \
--claim-mode rwo \ 
--claim-size 1Gi \ 
--mount-path /tmp/data \ 
--claim-name my-data-claim 
deployment.apps/my-deployment volume updated

The preceding command updates the deployment to add the volumes and volumeMounts sections, as shown in the following excerpt:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
  ...output omitted...
spec:
  ...output omitted...
  template:
    ...output omitted...
    spec:
      containers:
      - image: registry.ocp4.example.com:8443/rhel8/mysql-80
        ports:
        - containerPort: 3306
          protocol: TCP
        ...output omitted...
        volumeMounts:
        - mountPath: /tmp/data
          name: nfs-volume-storage
      ...output omitted...
      volumes:
      - name: nfs-volume-storage
        persistentVolumeClaim:
          claimName: my-data-claim
...output omitted...

One use for volumes is mounting pre-defined data into one or more pods without having to rebuild the container image. In such cases, you might not want to store new data from your application.

The following list includes example uses for ephemeral volumes:

Configuration maps and secrets can store file contents and can be mounted as a volume into a pod. In addition, you can create both configuration maps and secrets from local files.

Refer to previous sections for details on commands to create and mount configuration maps.

As their name suggests, stateful sets are intended for stateful applications. Unlike deployments, pods within stateful sets are guaranteed to have a predictable identifier (ID) for each pod. For example, three replicas for the redis stateful set might use the redis-0, redis-1, and redis-2 pod names. This is useful for routing requests to each pod, or discovering new pods that must join to an existing cluster. In addition, each replica pod created by a stateful set has a dedicated PVC.

Besides addressing storage, stateful sets also provide a means to separate networking amongst stateful replicas. A stateful application might require that each replica is addressable separately from the others. For example, clustered database server replicas often designate one as the primary and the others as secondary. Similarly, the replicas might need to communicate with each of the others to establish a quorum.

The following manifest outlines an example stateful set that includes the template for creating the PVC for each pod.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: my-stateful-app 
spec:
  ...output omitted...
  replicas: 3
  serviceName: my-stateful-app 
  template:
    ...output omitted...
    spec:
      containers:
        - image: my-app-image:latest
          name: mysql-80
          ports:
            - containerPort: 80
              protocol: TCP
          volumeMounts: 
            - name: app-pvc
              mountPath: /var/lib/mysql
              subPath: mysql-db
      volumes: 
        - name: init-db-volume
          configMap:
            name: init-db-cm
  volumeClaimTemplates:
    - metadata: 
        name: app-pvc
      spec:
        accessModes: [ "ReadWriteOnce" ]
        storageClassName: nfs-storage
        resources:
          requests:
            storage: 1Gi

The following manifest outlines the headless service referenced in the preceding stateful set:

apiVersion: v1
kind: Service
metadata:
  name: my-stateful-app
spec:
  clusterIP: None
  selector:
  ...output omitted...

Because the service defines clusterIP as None, it is a headless service and it will not be load balanced. However, the individual replicas receive individual IP addresses at the pod level. Consequently, you can route requests to a specific pod by using the pod name, such as my-stateful-app-0.my-stateful-app in the preceding example. This is often necessary in a stateful application as the replica pods are not interchangeable.