Bookmark this page

Guided Exercise: Implementing NFS Shared Storage

In this exercise, you will implement file sharing storage in an instance to address common use cases.

Outcomes

You should be able to create and use a shared file system with Red Hat Ceph Storage as its storage provider.

As the student user on the workstation machine, use the lab command to prepare your system for this exercise.

This command ensures that all resources required for the exercise are present.

[student@workstation ~]$ lab storage-nfs start

Procedure 6.3. Instructions

This guided exercise is in two parts. In part 1 you use the CLI, and in part 2 you use the Dashboard.

  1. As the domain operator, operator1, use the manila type-create command to create the share type cephfstype. Set the value of driver_handles_share_servers to false.

    1. On workstation source the identity environment file for the operator1 user in the finance project.

      [student@workstation ~]$ source ~/operator1-finance-rc
      [student@workstation ~(operator1-finance)]$
    2. Use the manila type-create command to create the share type.

      [student@workstation ~(operator1-finance)]$ manila type-create cephfstype false
      +----------------------+--------------------------------------+
      | Property             | Value                                |
      +----------------------+--------------------------------------+
      | ID                   | 6cdeb359-a353-4d7d-b50a-a3b55b713338 |
      | Name                 | cephfstype                           |
      | Visibility           | public                               |
      | is_default           | -                                    |
      | required_extra_specs | driver_handles_share_servers : False |
      | optional_extra_specs |                                      |
      | Description          | None                                 |
      +----------------------+--------------------------------------+
  2. As the cloud user, developer1 in the finance project, create a new share called finance-share1. The share should be 1 GB in size.

    1. Source the environment file for developer1-finance-rc

      [student@workstation ~(operator1-finance)]$ source ~/developer1-finance-rc
      [student@workstation ~(developer1-finance)]$
    2. Use the manila create command to create a share of type cephfs. Confirm that the status changes from creating to available.

      [student@workstation ~(developer1-finance)]$ manila create \
      > --name finance-share1 --share-type cephfstype cephfs 1
      +----------------------+--------------------------------------+
      | Property             | Value                                |
      +----------------------+--------------------------------------+
      | id                   | d2ad3b20-d0a2-4734-91de-f425f9964189 |
      | size                 | 1                                    |
      | availability_zone    | None                                 |
      | created_at           | 2020-07-02T09:07:11.000000           |
      | status               | creating                             |
      | name                 | finance-share1                       |
      ...output omitted...
      [student@workstation ~(developer1-finance)]$ manila list \
      > --columns Name,'Share Proto',Status,'Share Type Name'
      +----------------+-------------+-----------+-----------------+
      | Name           | Share Proto | Status    | Share Type Name |
      +----------------+-------------+-----------+-----------------+
      | finance-share1 | CEPHFS      | available | cephfstype      |
      +----------------+-------------+-----------+-----------------+
  3. Launch an instance finance-server6 in the finance project that uses two network interface cards. The first network interface card is connected to the internal tenant network finance-network1. The second network interface card is connected to the storage provider network provider-storage allowing the instance to directly reach the network of the shared file system (share). This instance will be used to access the share finance-share1.

    1. As developer1, launch the instance using the default flavor, the rhel8 image, the example-keypair key pair, the finance-network1 network, the provider-storage second network and the user data file /home/student/manila/user-data.file.

      [student@workstation ~(developer1-finance)]$ openstack server create \
      > --flavor default \
      > --image rhel8 \
      > --key-name example-keypair \
      > --nic net-id=finance-network1 \
      > --nic net-id=provider-storage \
      > --user-data /home/student/manila/user-data.file \
      > finance-server6 --wait
      ...output omitted...
    2. Add the available floating IP address to the instance.

      [student@workstation ~(developer1-finance)]$ openstack floating ip \
      > list -c 'Floating IP Address'
      +---------------------+
      | Floating IP Address |
      +---------------------+
      | 172.25.250.122      |
      +---------------------+
      [student@workstation ~(developer1-finance)]$ openstack server add \
      > floating ip finance-server6 172.25.250.122
    3. Confirm that the instance is reachable from workstation.

      [student@workstation ~(developer1-finance)]$ ping -c3 172.25.250.122
      PING 172.25.250.122 (172.25.250.122) 56(84) bytes of data.
      64 bytes from 172.25.250.122: icmp_seq=1 ttl=63 time=1.35 ms
      64 bytes from 172.25.250.122: icmp_seq=2 ttl=63 time=0.677 ms
      64 bytes from 172.25.250.122: icmp_seq=3 ttl=63 time=1.06 ms
      
      --- 172.25.250.122 ping statistics ---
      3 packets transmitted, 3 received, 0% packet loss, time 35ms
      rtt min/avg/max/mdev = 0.677/1.030/1.350/0.275 ms
  4. Create the exclusive cephx user client.cloud-user to access the Ceph-backed share finance-share1. Allow the client.cloud-user cephx user to read from and write to the share. The OpenStack Shared File Systems service uses the client.manila cephx user to authenticate as while communicating with the Ceph cluster. You are going to use the cephx authentication mechanism to enable access to the Ceph-backed share finance-share1.

    1. Log in to controller0 as the root user.

      [student@workstation ~(developer1-finance)]$ ssh root@controller0
      [root@controller0 ~]#
    2. Create the cephx user client.cloud-user. Save the key ring to /root/cloud-user.keyring.

      [root@controller0 ~]# podman exec -t \
      > ceph-mon-controller0 ceph --name=client.manila \
      > --keyring=/etc/ceph/ceph.client.manila.keyring \
      > auth get-or-create client.cloud-user > /root/cloud-user.keyring
    3. Confirm that the keyring was created correctly and contains the cloud-user key. Then log off from controller0.

      [root@controller0 ~]# cat /root/cloud-user.keyring
      [client.cloud-user]
          key = AQB4yf1ethSBMhAAm54S7XJay9PogZCl52nBGg==
      [root@controller0 ~]# logout
      [student@workstation ~(developer1-finance)]$
  5. Copy the ceph.conf and cloud-user.keyring files to the finance-server6 instance.

    1. Copy the ceph.conf and cloud-user.keyring files to the manila folder on workstation.

      [student@workstation ~(developer1-finance)]$ cd manila
      [student@workstation manila(developer1-finance)]$ scp \
      > root@controller0:{cloud-user.keyring,/etc/ceph/ceph.conf} .
      cloud-user.keyring    100%   70   107.1KB/s   00:00
      ceph.conf             100%  941     1.4MB/s   00:00
    2. Copy the ceph.conf and cloud-user.keyring from workstation to finance-server6. Return to the students home directory.

      [student@workstation manila(developer1-finance)]$ scp \
      > {cloud-user.keyring,ceph.conf} cloud-user@172.25.250.122:
      Warning: Permanently added '172.25.250.122' (ECDSA) to the list of known hosts.
      cloud-user.keyring    100%   70    43.5KB/s   00:00
      ceph.conf             100%  941   729.7KB/s   00:00
      [student@workstation manila(developer1-finance)]$ cd
      [student@workstation ~(developer1-finance)]$
  6. Add and verify the access rights to finance-share1 for client.cloud-user. Confirm the state of the share is active.

    [student@workstation ~(developer1-finance)]$ manila access-allow \
    > finance-share1 cephx cloud-user
    +--------------+--------------------------------------+
    | Property     | Value                                |
    +--------------+--------------------------------------+
    | id           | cb7c11f1-b710-45b9-8762-a34ae334fc2b |
    | share_id     | 4e099421-8733-4cf2-9af6-7159b7d24a37 |
    | access_level | rw                                   |
    | access_to    | cloud-user                           |
    | access_type  | cephx                                |
    | state        | queued_to_apply                      |
    | access_key   | None                                 |
    | created_at   | 2020-07-02T12:05:19.000000           |
    | updated_at   | None                                 |
    | metadata     | {}                                   |
    +--------------+--------------------------------------+
    [student@workstation ~(developer1-finance)]$ manila access-list \
    > finance-share1 --columns access_to,access_level,state
    +------------+--------------+--------+
    | Access_To  | Access_Level | State  |
    +------------+--------------+--------+
    | cloud-user | rw           | active |
    +------------+--------------+--------+
  7. Access finance-share1 from finance-server6. Create a mount point called /mnt/ceph for the share.

    1. Open an SSH session to finance-server6 as cloud-user.

      [student@workstation ~(developer1-finance)]$ ssh cloud-user@172.25.250.122
      [cloud-user@finance-server6 ~]$
    2. Create the mount point /mnt/ceph.

      [cloud-user@finance-server6 ~]$ sudo -i
      [root@finance-server6 ~]# mkdir /mnt/ceph
  8. Open another terminal window on workstation. As the developer1 cloud user, determine the export location of finance-share1.

    [student@workstation ~]$ source ~/developer1-finance-rc
    [student@workstation ~(developer1-finance)]$ manila share-export-location-list \
    > finance-share1 --columns Path
    +------------------------------------------------------------------------+
    | Path                                                                   |
    +------------------------------------------------------------------------+
    | 172.24.3.1:6789:/volumes/_nogroup/019cc044-1f71-43d5-bd6d-2b30817b1e57 |
    +------------------------------------------------------------------------+
  9. On finance-server6 enable the Ceph repositories on finance-server6 and install the ceph-fuse package.

    [root@finance-server6 ~]# curl -s -f \
    > -o /etc/yum.repos.d/ceph.repo http://materials.example.com/ceph.repo
    [root@finance-server6 ~]# yum install ceph-fuse
    ...output omitted...
    Is this ok [y/d/N]: y
    ...output omitted...
    Complete!
  10. Using the export path mount the share on /mnt/ceph. Verify that the share is successfully mounted.

    [root@finance-server6 ~]# ceph-fuse /mnt/ceph/ \
    > --id=cloud-user --conf=/home/cloud-user/ceph.conf \
    > --keyring=/home/cloud-user/cloud-user.keyring \
    > --client-mountpoint=/volumes/_nogroup/cea022a9-c00c-4003-b6f3-8fea2a49bd5f
     2020-07-02 08:27:39.600 7f22c17d7040 -1 init, newargv = 0x555b64a4e460 newargc=7
     ceph-fuse[11810]: starting ceph client
     ceph-fuse[11810]: starting fuse
    [root@finance-server6 ~]# df -Th
    ...output omitted...
    ceph-fuse      fuse.ceph-fuse  1.0G     0  1.0G   0% /mnt/ceph
  11. To confirm that finance-share1 is readable and writable create a text file in the /mnt/ceph directory. Log out of finance-server6.

    [root@finance-server6 ~]# echo hello > /mnt/ceph/hello.txt
    [root@finance-server6 ~]# cat /mnt/ceph/hello.txt
    hello
    [root@finance-server6 ~]# logout
    [cloud-user@finance-server6 ~]$ logout
    [student@workstation ~(developer1-finance)]$

    Note

    Use the Dashboard to perform the following steps.

  12. As developer1, navigate and verify the networking and shares.

    1. Log in to the Dashboard at http://dashboard.overcloud.example.com using Example for the domain, developer1 for the user, and redhat as the password. Click Project in the upper right corner and select finance.

    2. Navigate to ProjectNetworkNetwork Topology. Note the network connections. The instance finance-server6 is attached to finance-network1 and provider-storage network. The router is attached to both finance-network1 and the provider-datacentre network. If the router was not attached to the provider network, it would have been impossible to assign a floating IP address to the instance.

    3. Navigate to ProjectShareShares. Note that there is one share finance-share1. In the action menu you can edit, manage, and delete the share. You cannot however, create the ceph keyring or the share type in the Dashboard. Those must be created using the CLI.

Finish

On the workstation machine, use the lab command to complete this exercise. This is important to ensure that resources from previous exercises do not impact upcoming exercises.

[student@workstation ~]$ lab storage-nfs finish

This concludes the guided exercise.

Revision: cl110-16.1-4c76154