After completing this section, you should be able to identify and configure the primary settings for the overall Red Hat Ceph Storage cluster.

All Red Hat Ceph Storage cluster configurations contain these required definitions:

Ceph configuration settings use unique names that consist of lowercase character words connected with underscores.

Every Ceph daemon, process, and library accesses its configuration from one of these sources:

The monitor (MON) nodes manage a centralized configuration database. On startup, Ceph daemons parse configuration options that are provided via command-line options, environment variables, and the local cluster configuration file. The daemons then contact the MON cluster to retrieve configuration settings that are stored in the centralized configuration database.

Red Hat Ceph Storage 5 deprecates the ceph.conf cluster configuration file, making the centralized configuration database the preferred way to store configuration settings.

Each Ceph node stores a local cluster configuration file. The default location of the cluster configuration file is /etc/ceph/ceph.conf. The cephadm tool creates an initial Ceph configuration file with a minimal set of options.

The configuration file uses an INI file format, containing several sections that include configuration for Ceph daemons and clients. Each section has a name that is defined with the [name] header, and one or more parameters that are defined as a key-value pair.

[name]
parameter1 = value1
parameter2 = value2

Use a hash sign (#) or semicolon (;) to disable settings or to add comments.

Bootstrap a cluster with custom settings by using a cluster configuration file. Use the cephadm boostrap command with the --config option to pass the configuration file.

[root@node ~]# cephadm bootstrap --config ceph-config.yaml

[mon]
# Settings for all mon daemons

[mon.serverc]
# Settings that apply to the specific MON daemon running on serverc

The MON cluster manages and stores the centralized configuration database on the MON nodes. You can either change a setting temporarily, until the daemons restart, or configure a setting permanently and store it in the database. You can change most configuration settings while the cluster is running.

Use ceph config commands to query the database and view configuration information.

Use the assimilate-conf subcommand to apply configuration from a file to a running cluster. This process recognizes and applies the changed settings from the configuration file to the centralized database. This command is useful to import custom settings from a previous storage cluster to a new one. Invalid or unrecognized options display on standard output, and require manual handling. Redirect screen output to a file using the -o output-file.

[ceph: root@node /]# ceph config assimilate-conf -i ceph.conf

Some options provide the information needed to start the cluster. MON nodes read the monmap to find other MONs and establish a quorum. MON nodes read the ceph.conf file to identify how to communicate with other MONs.

The mon_host option lists cluster monitors. This option is essential and cannot be stored in the configuration database. To avoid using a cluster configuration file, Ceph clusters support using DNS service records to provide the mon_host list.

The local cluster configuration file can contain other options to fit your requirements.

Service configuration files are YAML files that bootstrap a storage cluster and additional Ceph services. The cephadm tool orchestrates the service deployment, sizing, and placement by balancing the running daemons in the cluster. Various parameters can deploy services such as OSDs or MONs in a more defined manner.

An example service configuration file follows.

service_type: mon
placement:
  host_pattern: "mon*"
  count: 3
---
service_type: osd
service_id: default_drive_group
placement:
  host_pattern: "osd*"
data_devices:
  all: true

Use the cephadm bootstrap --apply-spec command to apply the service configurations from the specified file.

[root@node ~]# cephadm bootstrap --apply-spec service-config.yaml

You can change most cluster configuration settings while running. You can temporarily change a configuration setting while the daemon is running.

The ceph tell $type.$id config command temporarily overrides configuration settings, and requires that both the MONs and the daemon being configured are running.

Run this command from any cluster host configured to run ceph commands. Settings that are changed with this command revert to their original settings when the daemon restarts.

The ceph tell $type.$id config command can also accept wildcards to get or set the value on all daemons of the same type. For example, ceph tell osd.* config get debug_ms displays the value of that setting for all OSD daemons in the cluster.

You can use the ceph daemon $type.$id config command to temporarily override a configuration setting. Run this command on the cluster node where the setting is required.

The ceph daemon does not need to connect through the MONs. The ceph daemon command will still function even when the MONs are not running, which can be useful for troubleshooting.