CL260 - ch07s03

Bookmark this page

Providing iSCSI Block Storage

Objectives

After completing this section, you should be able to configure the Ceph iSCSI Gateway to export RADOS Block Devices using the iSCSI protocol, and configure clients to use the iSCSI Gateway.

Describing the Ceph iSCSI Gateway

Red Hat Ceph Storage 5 can provide highly available iSCSI access to RADOS block device images stored in the cluster. The iSCSI protocol allows clients (initiators) to send SCSI commands to storage devices (targets) over TCP/IP networks. Each initiator and target is uniquely identified by an iSCSI qualified name (IQN). Clients with standard iSCSI initiators can access cluster storage without requiring native Ceph RBD client support.

The Linux I/O target kernel subsystem runs on every iSCSI gateway to support the iSCSI protocol. Previously called LIO, the ISCSI target subsystem is now called TCM, or the Target Core Mod. The TCM subsystem utilizes a user-space pass-through (TCMU) to interact with the Ceph librbd library to expose RBD images to iSCSI clients.

iSCSI-specific OSD Tuning

Object Storage Devices (OSDs) and Monitors (MONs) do not require any iSCSI-specific server settings. To limit client SCSI time outs, reduce the delay timeout setting that the cluster uses to detect a failing OSD.

In the cephadm shell, run the ceph tell <daemon_type>.<id> config set command to set the timeout parameters.

[root@node ~]# ceph config set osd osd_heartbeat_interval 5
[root@node ~]# ceph config set osd osd_heartbeat_grace 20
[root@node ~]# ceph config set osd osd_client_watch_timeout 15

Deploying an iSCSI Gateway

You can deploy the iSCSI gateway on dedicated nodes or collocated with the OSDs. Meet the following prerequisites before deploying a Red Hat Ceph Storage iSCSI gateway:

Install the iSCSI gateway nodes with Red Hat Enterprise Linux 8.3 or later.
Have an operational cluster running Red Hat Ceph Storage 5 or later.
Have 90 MiB of RAM available for each RBD image exposed as a target on iSCSI gateway nodes.
Open TCP ports 3260 and 5000 on the firewall on each Ceph iSCSI node.
Create a new RADOS block device or use an existing, available device.

Create a Configuration File

To deploy iSCSI gateway nodes, use the cephadm shell to create a configuration file called iscsi-gateway.yaml in the /etc/ceph/ directory. The file should display as follows:

service_type: iscsi
service_id: iscsi
placement:
  hosts:
  - serverc.lab.example.com
  - servere.lab.example.com
spec:
  pool: iscsipool1
  trusted_ip_list: "172.25.250.12,172.25.250.14"
  api_port: 5000
  api_secure: false
  api_user: admin
  api_password: redhat

Apply the Specification File and Deploy the iSCSI Gateway

Run the ceph orch apply command to implement the configuration by using the -i option to use a specification file.

[ceph: root@node /]# ceph orch apply -i /etc/ceph/iscsi-gateway.yaml
Scheduled iscsi.iscsi update...

List the gateways and verify that they are present.

[ceph: root@node /]# ceph dashboard iscsi-gateway-list
{"gateways": {"serverc.lab.example.com": {"service_url": "http://admin:redhat@172.25.250.12:5000"}, "servere.lab.example.com": {"service_url": "http://admin:redhat@172.25.250.14:5000"}}}

Open a web browser and log in to the Ceph Dashboard as a user with administrative privileges. In the Ceph Dashboard web UI, click Block → iSCSI to display the iSCSI Overview page.

Figure 7.3: iSCSI gateway Dashboard page

After the Ceph Dashboard is configured to access the iSCSI gateway APIs, use it to manage iSCSI targets. Use the Ceph Dashboard to create, view, edit, and delete iSCSI targets.

Configure an iSCSI Target

Use the Ceph Dashboard or the ceph-iscsi gwcli utility to configure an iSCSI target.

The Using gwcli to add more iSCSI gateways section of the Block Device Guide for Red Hat Ceph Storage 5 provides detailed instructions on how to manage iSCSI targets using the ceph-iscsi gwcli utility.

These are example steps to configure an iSCSI target from the Ceph Dashboard.

Log in to the Dashboard.
On the navigation menu, click Block → iSCSI.
Click the Targets tab.
Select Create from the Create list.
In the Create Target window, set the following parameters:
1. Modify the Target IQN (optional).
2. Click +Add portal and select the first of at least two gateways.
3. Click +Add image and select an image for the target to export.
4. Click Create Target.

Figure 7.4: iSCSI Targets page

Configure an iSCSI Initiator

Configuring an iSCSI initiator to communicate with the Ceph iSCSI gateway is the same as for any industry-standard iSCSI gateway. For RHEL 8, install the iscsi-initiator-utils and the device-mapper-multipath packages. The iscsi-initiator-utils package contains the utilities required to configure an iSCSI initiator. When using more than one iSCSI gateway, configure clients for multipath support to failover between gateways by using the cluster's iSCSI targets.

Note

A system might be able to access the same storage device through multiple different communication paths, whether those are using Fibre Channel, SAS, iSCSI, or some other technology. Multipathing allows you to configure a virtual device that can use any of these communication paths to access your storage. If one path fails, then the system automatically switches to use one of the other paths instead.

If deploying a single iSCSI gateway for testing, skip the multipath configuration.

These example steps configure an iSCSI initiator to use multipath support and to log in to an iSCSI target. Configure your client's Challenge-Handshake Authentication Protocol (CHAP) user name and password to log in to the iSCSI targets.

Install the iSCSI initiator tools.

[root@node ~]# yum install iscsi-initiator-utils

Configure multipath I/O.

Install the multipath tools.

[root@node ~]# yum install device-mapper-multipath

Enable and create a default multipath configuration.

[root@node ~]# mpathconf --enable --with_multipathd y

Add the following to the /etc/multipath.conf file.

devices {
    device {
        vendor                 "LIO-ORG"
        hardware_handler       "1 alua"
        path_grouping_policy   "failover"
        path_selector          "queue-length 0"
        failback               60
        path_checker           tur
        prio                   alua
        prio_args              exclusive_pref_bit
        fast_io_fail_tmo       25
        no_path_retry          queue
    }
}

Restart the multipathd service.

[root@node ~]# systemctl reload multipathd

If required for your configuration, set CHAP authentication.
- Update the CHAP user name and password to match your iSCSI gateway configuration in the /etc/iscsi/iscsid.conf file.
```
node.session.auth.authmethod = CHAP
node.session.auth.username = user
node.session.auth.password = password
```

Discover and log in to the iSCSI portal, and then view targets and their multipath configuration.

Discover the iSCSI portal.

[root@node ~]# iscsiadm -m discovery -t st -p 10.30.0.210
10.30.0.210:3260,1 iqn.2001-07.com.ceph:1634089632951
10.30.0.133:3260,2 iqn.2001-07.com.ceph:1634089632951

[root@node ~]# iscsiadm -m node -T iqn.2001-07.com.ceph:1634089632951 -l
Logging in to [iface: default, target: iqn.2001-07.com.ceph:1634089632951, portal: 10.30.0.210,3260]
Logging in to [iface: default, target: iqn.2001-07.com.ceph:1634089632951, portal: 10.30.0.133,3260]
Login to [iface: default, target: iqn.2001-07.com.ceph:1634089632951, portal: 10.30.0.210,3260] successful.
Login to [iface: default, target: iqn.2001-07.com.ceph:1634089632951, portal: 10.30.0.133,3260] successful.

Verify any attached SCSI targets.

[root@node ~]# lsblk
NAME     MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sda        8:0    0    1G  0 disk
└─mpatha 253:0    0    1G  0 mpath
sdb        8:16   0    1G  0 disk
└─mpatha 253:0    0    1G  0 mpath
vda      252:0    0   10G  0 disk

Use the multipath command to show devices set up in a failover configuration with a priority group for each path.
```
[root@node ~]# multipath -ll
mpatha (3600140537b026aa91c844138da53ffe7) dm-0 LIO-ORG,TCMU device
size=1.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='queue-length 0' prio=50 status=active
| `- 2:0:0:0 sda 8:0  active ready running
`-+- policy='queue-length 0' prio=10 status=enabled
  `- 3:0:0:0 sdb 8:16 active ready running
```
Note
If logging in to an iSCSI target through a single iSCSI gateway, then the system creates a physical device for the iSCSI target (for example, /dev/sdX). If logging in to an iSCSI target through multiple iSCSI gateways with Device Mapper multipath, then the Device Mapper creates a multipath device (for example, /dev/mapper/mpatha).

References

For more information, refer to the The Ceph iSCSI Gateway chapter in the Block Device Guide for Red Hat Ceph Storage 5 at https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/5/html-single/block_device_guide/index#the-ceph-iscsi-gateway

For more information, refer to the Management of iSCSI functions using the Ceph dashboard section in the _Dashboard Guide for Red Hat Ceph Storage 5 at https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/5/html-single/dashboard_guide/index#management-of-iscsi-functions-on-the-ceph-dashboard

Further information, refer to the Device Mapper Multipath configuration for Red Hat Enterprise Linux 8 is available at Configuring Device Mapper Multipath at https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/configuring_device_mapper_multipath/

Discuss Cloud Storage with Red Hat Ceph Storage

Go to community

Red Hat Ceph Storage for OpenStack (CL260)

Haley_Ruccio

3 sie 2023

Build, expand and maintain cloud-scale, clustered storage for your applications with Red Hat Ceph StorageCloud Storage with Red Hat Ceph Storage (CL260) is designed for storage administrators and cloud operators who deploy Red Hat Ceph Storage in a production data center environment or as a component of a Red Hat OpenStack Platform or OpenShift Container Platform infrastructure. Learn how to deploy, manage, and scale a Ceph storage cluster to provide hybrid storage resources, including Amazon S3 and OpenStack Swift-compatible object storage, Ceph-native and iSCSI-based block storage, and shared file storage. This course is based on Red Hat Ceph Storage 5.0.Course summaryDeploy and manage a Red Hat Ceph Storage cluster on commodity servers.Perform common management operations using the web-based management interface.Create, expand, and control access to storage pools provided by the Ceph cluster.Access Red Hat Ceph Storage from clients using object, block, and file-based methods.Analyze and tune Red Hat Ceph Storage performance.Integrate Red Hat OpenStack Platform image, object, block, and file storage with a Red Hat Ceph Storage cluster.Integrate OpenShift Container Platform with a Red Hat Ceph Storage cluster.Target AudienceThis course is intended for storage administrators and cloud operators who want to learn how to deploy and manage Red Hat Ceph Storage on servers in an enterprise data center or within a Red Hat OpenStack Platform or OpenShift Container Platform environment.Developers writing applications that use cloud-based storage will learn the distinctions of various storage types and client access methods.Recommended trainingTake our free assessment to gauge whether this offering is the best fit for your skills.Red Hat Certified System Administrator (RHCSA) certification, or equivalent experience.For candidates that have not earned an RHCSA or equivalent, confirmation of the correct skill set knowledge can be obtained by taking the online skills assessment.Some experience with storage administration is recommended but not required.Technology considerationsThis course does not have any special technical requirements.This course is not intended for BYOD.Internet access is recommended.

Welcome to the Red Hat Ceph Storage for OpenStack (CL260) group in the Red Hat Learning Community!

cschunke

31 lip 2023

We are excited to launch a space dedicated to the Red Hat Training course CL260! To gain the most value from this group - click the "Join Group" button in the upper right hand corner of the group home page.We encourage group members to collaborate in this group to discuss topics, ask questions, share best practices and tips, provide course feedback, and share their accomplishments as it relates to CL260.Read more about Red Hat Ceph Storage for OpenStack here.

381

Revision: cl260-5.0-29d2128