Cloud Storage with Red Hat Ceph Storage

Procedure 3.2. Instructions

1. Log in to clienta as the admin user and use sudo to run the cephadm shell.

   [student@workstation ~]$ ssh admin@clienta
   [admin@clienta ~]$ sudo cephadm shell
   [ceph: root@clienta /]#

2. Use the ceph status command to view the cluster quorum status.

   [admin@clienta ~]$ ceph status
       id:     472b24e2-1821-11ec-87d7-52540000fa0c
       health: HEALTH_OK
   
     services:
       mon: 4 daemons, quorum serverc.lab.example.com,serverd,servere,clienta (age 1h)
   ...output omitted...

3. View the MON map.

   [ceph: root@clienta /]# ceph mon dump
   epoch 4
   fsid 472b24e2-1821-11ec-87d7-52540000fa0c
   last_changed 2021-09-20T01:41:44.138014+0000
   created 2021-09-18T01:39:57.614592+0000
   min_mon_release 16 (pacific)
   election_strategy: 1
   0: [v2:172.25.250.12:3300/0,v1:172.25.250.12:6789/0] mon.serverc.lab.example.com
   1: [v2:172.25.250.13:3300/0,v1:172.25.250.13:6789/0] mon.serverd
   2: [v2:172.25.250.14:3300/0,v1:172.25.250.14:6789/0] mon.servere
   3: [v2:172.25.250.10:3300/0,v1:172.25.250.10:6789/0] mon.clienta
   dumped monmap epoch 4

4. View the value of the mon_host setting for the serverc MON.

   [ceph: root@clienta /]# ceph config show mon.serverc.lab.example.com mon_host
   [v2:172.25.250.10:3300/0,v1:172.25.250.10:6789/0] [v2:172.25.250.12:3300/0,v1:172.25.250.12:6789/0] [v2:172.25.250.13:3300/0,v1:172.25.250.13:6789/0] [v2:172.25.250.14:3300/0,v1:172.25.250.14:6789/0]

5. View the MON IP, port information, and quorum status in the MON stats.

   [ceph: root@clienta /]# ceph mon stat
   e4: 4 mons at {clienta=[v2:172.25.250.10:3300/0,v1:172.25.250.10:6789/0], serverc.lab.example.com=[v2:172.25.250.12:3300/0,v1:172.25.250.12:6789/0], serverd=[v2:172.25.250.13:3300/0,v1:172.25.250.13:6789/0], servere=[v2:172.25.250.14:3300/0,v1:172.25.250.14:6789/0]}, election epoch 66, leader 0 serverc.lab.example.com, quorum 0,1,2,3 serverc.lab.example.com,serverd,servere,clienta

6. Use the ceph auth ls command to view the cluster authentication settings.

   [ceph: root@clienta /]# ceph auth ls
   osd.0
   	key: AQDiQ0Vh905yKBAAI0l2a+53fcCbNVBeJAeJvQ==
   	caps: [mgr] allow profile osd
   	caps: [mon] allow profile osd
   	caps: [osd] allow *
   ...output omitted...
   client.admin
   	key: AQBsQ0Vhe3UqMhAA+44H1rvTDjgBPvdDGXBK+A==
   	caps: [mds] allow *
   	caps: [mgr] allow *
   	caps: [mon] allow *
   	caps: [osd] allow *
   ...output omitted...
   mgr.serverc.lab.example.com.xgbgpo
   	key: AQBtQ0VhFCRnDhAAbnmlnzHbcmq2akAjlZuigg==
   	caps: [mds] allow *
   	caps: [mon] profile mgr
   	caps: [osd] allow *
   ...output omitted...

7. Export the client.admin key ring.

   Note

   The admin key ring is stored by default in the /etc/ceph/ceph.client.admin.keyring file.

   [ceph: root@clienta /]# ceph auth get client.admin -o /tmp/adminkey
   exported keyring for client.admin
   [ceph: root@clienta /]# cat /tmp/adminkey
   [client.admin]
   	key = AQBsQ0Vhe3UqMhAA+44H1rvTDjgBPvdDGXBK+A==
   	caps mds = "allow *"
   	caps mgr = "allow *"
   	caps mon = "allow *"
   	caps osd = "allow *"

8. Verify the space that the MON database uses on serverc. Set the option to compact the MON database on start. Use ceph orch to restart the MON daemons and wait for the cluster to reach a healthy state. Verify again the space of the MON database on serverc.

   Note

   Your cluster is expected to have a different size than in the examples.

   1. Verify the space that the MON database uses on serverc. The name of the fsid folder inside /var/lib/ceph can be different in your environment.

      [ceph: root@clienta /]# exit
      exit
      [admin@clienta ~]$ ssh serverc sudo du -sch \
        /var/lib/ceph/472b...a0c/mon.serverc.lab.example.com/store.db/
      admin@serverc's password: redhat
      74M	/var/lib/ceph/472b...a0c/mon.serverc.lab.example.com/store.db/
      74M	total

   2. Set the option to compact the MON database on start. Use ceph orch to restart the MON daemons, then wait for the cluster to reach a healthy state. This process can take many seconds.

      [admin@clienta ~]$ sudo cephadm shell
      [ceph: root@clienta /]# ceph config set mon mon_compact_on_start true
      [ceph: root@clienta /]# ceph orch restart mon
      Scheduled to restart mon.clienta on host 'clienta.lab.example.com'
      Scheduled to restart mon.serverc.lab.example.com on host 'serverc.lab.example.com'
      Scheduled to restart mon.serverd on host 'serverd.lab.example.com'
      Scheduled to restart mon.servere on host 'servere.lab.example.com'
      [ceph: root@clienta /]# ceph health
      HEALTH_OK

   3. Exit the cephadm shell, then verify the current space of the MON database on serverc.

      [ceph: root@clienta /]# exit
      exit
      [admin@clienta ~]$ ssh serverc sudo du -sch \
        /var/lib/ceph/472b...a0c/mon.serverc.lab.example.com/store.db/
      admin@serverc's password: redhat
      71M	/var/lib/ceph/472b...a0c/mon.serverc.lab.example.com/store.db/
      71M	total

      Note

      The MON database compact process can take a few minutes depending on the size of the database. If the file size is bigger than before, then wait a few seconds until the file is compacted.

9. Return to workstation as the student user.

   [admin@clienta ~]$ exit
   [student@workstation ~]$