After completing this section, you should be able to deploy a RADOS Gateway to provide clients with access to Ceph object storage.

Object storage stores data as discrete items, each individually called an object. Unlike files in a file system, objects are not organized in a tree of directories and subdirectories. Instead, objects are stored in a flat namespace. Each object is retrieved by using the object's unique object ID, also known as an object key.

Applications do not use normal file-system operations to access object data. Instead, applications access a REST API to send and receive objects. Red Hat Ceph Storage supports the two most common object APIs, Amazon S3 (Simple Storage Service) and OpenStack Swift (OpenStack Object Storage).

Amazon S3 calls the flat namespace for object storage a bucket while OpenStack Swift calls it a container. Because a namespace is flat, neither buckets nor containers can be nested. Ceph typically uses the term bucket, as does this lecture.

A single user account can be configured for access to multiple buckets on the same storage cluster. Buckets can each have different access permissions and be used to store objects for different use cases.

The advantage of object storage is that it is easy to use, expand, and scale. Because each object has a unique ID, it can be stored or retrieved without the user knowing the object's location. Without the directory hierarchy, relationships between objects are simplified.

Objects, similar to files, contain a binary data stream and can grow to arbitrarily large sizes. Objects also contain metadata about the object data, and natively support extended metadata information, typically in the form of key-value pairs. You can also create your own metadata keys and store custom information in the object as key values.

The RADOS Gateway, also called the Object Gateway (RGW), is a service that provides access to the Ceph cluster for clients using standard object storage APIs. The RADOS Gateway supports both the Amazon S3 and OpenStack Swift APIs.

The core daemon, radosgw, is built on top of the librados library. The daemon provides a web service interface, based on the Beast HTTP, WebSocket, and networking protocol library, as a front-end to handle API requests.

The radosgw is a client to Red Hat Ceph Storage that provides object access to other client applications. Client applications use standard APIs to communicate with the RADOS Gateway, and the RADOS Gateway uses librados module calls to communicate with the Ceph cluster.

Figure 8.1: RADOS Gateway service architecture

The RADOS Gateway provides the radosgw-admin utility for creating users for using the gateway. These users can only access the gateway, and are not cdephx users with direct access to the storage cluster. RADOS Gateway clients authenticate using these gateway user accounts when submitting Amazon S3 or OpenStack Swift API requests. After a gateway user is authenticated by the RADOS Gateway, the gateway uses cephx credentials to authenticate to the storage cluster to handle the object request. Gateway users can also be managed by integrating an external LDAP-based authentication service.

The RADOS Gateway service automatically creates pools on a per-zone basis. These pools use placement group values from the configuration database and use the default CRUSH hierarchy. The default pool settings might not be optimal for a production environment.

The RADOS Gateway creates multiple pools for the default zone.

You can manually create pools with custom settings. Red Hat recommends using the zone name as a prefix for manually created pools, as in .<zone-name>.rgw.control. For example, using .us-east-1.rgw.buckets.data as a pool name when us-east-1 is the zone name.

The cephadm tool deploys RADOS Gateway services as a collection of daemons that manage a single cluster or a multisite deployment. Use the client.rgw.* section in the centralized configuration database to define parameters and characteristics for new RADOS Gateway daemons.

Use the Ceph Orchestrator to deploy or remove RADOS Gateway services. Use the Ceph Orchestrator with either the command-line interface or a service specification file.

[ceph: root@node /]# ceph orch apply rgw <service-name> [--realm=<realm>] \
[--zone=<zone>] [--port=<port>] --placement="<num-daemons> [<host1> ...]" \
[--unmanaged]

In this example, the Ceph Orchestrator deploys the my_rgw_service RADOS Gateway service with two daemons in a single cluster, and presents the service on port 80.

[ceph: root@node /]# ceph orch apply rgw my_rgw_service

If no configuration is defined in the client.rgw.* section or passed to the Ceph orchestrator at build time, then the deployment uses these default settings.

The following example YAML file contains common parameters defined for a RADOS Gateway deployment.

service_type: rgw
service_name: rgw_service_name
placement:
  count: 2
  hosts:
  - node01
  - node02
spec:
  rgw_frontend_port: 8080
  rgw_realm: realm_name
  rgw_zone: zone_name
  ssl: true
  rgw_frontend_ssl_certificate: |
    -----BEGIN PRIVATE KEY-----
    ...output omitted...
    -----END PRIVATE KEY-----
    -----BEGIN CERTIFICATE-----
    ...output omitted...
    -----END CERTIFICATE-----
  networks:
    - 172.25.200.0/24

In this example, a RGW service is created with similar parameters than the previous one, but now using the CLI.

[ceph: root@node /]# ceph orch apply rgw rgw_service_name --realm=realm_name \
--zone=zone_name --port 8080 --placement="2 node01 node 02" --ssl

Notice that in the service specification file, the parameter names for the realm, zone, and port are different than the used by the CLI. Some parameters such as the network to be used by RGW instances or the ssl certificate content can only be defined by using the service speficiation file.

The count parameter sets the number of RGW instances to be created on each server defined in the hosts parameter. If you create more than one instance, then the Ceph orchestrator sets the port of the first instance to the specified rgw_frontend_port or port value from. For each subsequent instance, the port value is increased by 1. Using the previous YAML file example, the service deployment creates:

Each instance has its own unique port enabled for access and creates the same responses to requests. Configure high availability for the RADOS Gateway by deploying a load-balancer service that presents a single service IP address and port.

[ceph: root@node /]# ceph config get client.rgw rgw_frontends
beast port=7480

[ceph: root@node /]# ceph config get client.rgw rgw_frontends
beast port=80+443s

The RADOS Gateway provides Beast embedded HTTP servers as a front-end. A Beast front-end uses the Boost.Beast library for HTTP parsing and the Boost.Asio library for asynchronous network I/O.

When the RADOS Gateway service load increases, you can deploy more RGW instances to support the workload. You can add instances in a single zone group deployment, but consider that each RGW instance has its own IP address and it can be difficult to balance requests to different instances in a single zone.

Instead, configure HAProxy and keepalived to balance the load across RADOS Gateway servers. HAProxy presents only one IP address and it balances the requests to all RGW instances. Keepalived ensures that the proxy nodes maintain the same presented IP address, independent of node availability.

You can configure the HAProxy service to use HTTPS. To enable HTTPS, generate SSL keys and certificates for the configuration. If you do not have a certificate from a Certificate Authority, then use a self-signed certificate.