These core OpenStack services provide storage resources in various formats and by multiple access methods. Cloud users deploy application VMs that consume these storage resources.

These additional core services provide overcloud installation, service container deployment, and the authentication support that are necessary to implement storage integration.

A dedicated Ceph implementation is the TripleO default, and is sufficient for most small, medium, and moderately large OpenStack installations. A storage operator has significant choices for service distribution across overcloud nodes by using composable node roles. Except where stated otherwise, these node roles are included by default in later RHOSP versions.

SectionFigure 13.2: An example overcloud with multiple node roles presents an example of overcloud nodes to implement different service roles in a simple overcloud.

Figure 13.2: An example overcloud with multiple node roles

The following node roles determine the services that are placed on storage nodes that handle data plane traffic and on the physical storage devices.

The CephStorage role is the default, and control plane services are expected to be installed on controller nodes.

When storage management traffic increases, controller nodes can become overloaded. The following node roles support various configurations and distributions of Ceph control plane services across multiple nodes. Coordinate controller node roles with role choices for storage nodes to ensure that all wanted control plane services are deployed.

The following node roles are not included by default in the RHOSP distribution, but are described in Red Hat online documentation. Use these roles to alleviate overloaded controller nodes by moving primary Ceph services to separate, dedicated nodes. These roles are commonly found in larger OpenStack installations with increased storage traffic requirements.

A Hyperconverged Infrastructure (HCI) node is a configuration with both compute and storage services and devices on the same node. This configuration can result in increased performance for heavy storage throughput applications. The default is the ComputeHCI role, which adds only OSDs to a compute node, effectively enlarging your dedicated Ceph cluster. Ceph control plane services remain on the controller nodes. The other node roles add various choices of control plane services to the hyperconverged node.

A Distributed Compute Node (DCN) is another form of hyperconverged node that is designed for use in remote data centers or branch offices that are part of the same OpenStack overcloud. For DCN, the overcloud deployment creates a dedicated Ceph cluster, with a minimum of three nodes, per remote site in addition to the dedicated Ceph cluster at the primary site. This architecture is not a stretch cluster configuration. Later DCN versions support installing the Glance in the remote location for faster local image access.

RHOSP overcloud installations have an undercloud node, which is referred to as the Director node in SectionFigure 13.1: A simple set of OpenStack services. TripleO installs overcloud from the Director node. The default orchestration templates for TripleO services are in the /usr/share/openstack-tripleo-heat-templates directory on the undercloud. When deploying OpenStack integrated with Ceph, the undercloud node becomes the Ansible controller and cluster administration host.

Because the default orchestration files are continuously being enhanced, you must not modify default template files in their original location. Instead, create a directory to store your custom environment files and parameter overrides. The following ceph-ansible-external.yaml environment file instructs TripleO to use the ceph-ansible client role to access a preexisting, external Ceph cluster. To override the default settings in this file, use a custom parameter file.

[stack@director ceph-ansible]$ cat ceph-ansible-external.yaml
 resource_registry:
  OS::TripleO::Services::CephExternal: ../../deployment/ceph-ansible/ceph-external.yaml

parameter_defaults:
  # NOTE: These example parameters are required when using CephExternal
  #CephClusterFSID: '4b5c8c0a-ff60-454b-a1b4-9747aa737d19'
  #CephClientKey: 'AQDLOh1VgEp6FRAAFzT7Zw+Y9V6JJExQAsRnRQ=='
  #CephExternalMonHost: '172.16.1.7, 172.16.1.8'

  # the following parameters enable Ceph backends for Cinder, Glance, Gnocchi and Nova
  NovaEnableRbdBackend: true
  CinderEnableRbdBackend: true
  CinderBackupBackend: ceph
  GlanceBackend: rbd
  # Uncomment below if enabling legacy telemetry
  # GnocchiBackend: rbd
  # If the Ceph pools which host VMs, Volumes and Images do not match these
  # names OR the client keyring to use is not called 'openstack',  edit the
  # following as needed.
  NovaRbdPoolName: vms
  CinderRbdPoolName: volumes
  CinderBackupRbdPoolName: backups
  GlanceRbdPoolName: images
  # Uncomment below if enabling legacy telemetry
  # GnocchiRbdPoolName: metrics
  CephClientUserName: openstack

  # finally we disable the Cinder LVM backend
  CinderEnableIscsiBackend: false

A TripleO deployment specifies a list of environment files for all of the overcloud services to be deployed, with an openstack overcloud deploy command. Before deployment, the openstack tripleo container image prepare command is used to determine all of the services that are referenced in the configuration, and prepare a list of the corrector containers to download and provide for the overcloud deployment. During the installation, Kolla is used to configure and start each service container on the correct nodes, as defined by the node roles.

For this external Ceph cluster example, TripleO needs a parameter file that specifies the real cluster parameters, to override the parameter defaults in the ceph-ansible-external.yaml file. This example parameter-overrides.yaml file is placed in your custom deployment files directory. You can obtain the key from the result of an appropriate ceph auth add client.openstack command.

      parameter_defaults:
        # The cluster FSID
        CephClusterFSID: '4b5c8c0a-ff60-454b-a1b4-9747aa737d19'
        # The CephX user auth key
        CephClientKey: 'AQDLOh1VgEp6FRAAFzT7Zw+Y9V6JJExQAsRnRQ=='
        # The list of Ceph monitors
        CephExternalMonHost: '172.16.1.7, 172.16.1.8, 172.16.1.9'

TripleO relies on the Bare Metal service to prepare nodes before installing them as Ceph servers. Disk devices, both physical and virtual, must be cleaned of all partition tables and other artifacts. Otherwise, Ceph refuses to overwrite the device, after determining that the device is in use. To delete all metadata from disks, and create GPT labels, set the following parameter in the /home/stack/undercloud.conf file on the undercloud. The Bare Metal service boots the nodes and cleans the disks each time the node status is set to available for provisioning.

clean_nodes=true