After completing this section, you should be able to configure the RADOS Gateway to provide access to object storage compatible with the Amazon S3 API, and manage objects stored using that API.

The Amazon S3 API enables developers to manage object storage resources using an Amazon S3 compatible interface. Applications implemented with the S3 API can inter-operate with other S3-compatible object storage services besides the RADOS Gateway, and migrate storage from other locations to your Ceph storage cluster. In a hybrid cloud environment, you can configure your applications to use different authentication keys, regions, and vendor services to mix private enterprise and public cloud resources and storage locations seamlessly using the same API.

The Amazon S3 interface defines the namespace in which objects are stored as a bucket. To access and manage objects and buckets using the S3 API, applications use RADOS Gateway users for authentication. Each user has an access key that identifies the user and a secret key that authenticates the user.

There are object and metadata size limits to consider when using the Amazon S3 API:

Create the needed RADOS Gateway users first, then use them to authenticate Amazon S3 clients with the gateway. Use the radosgw-admin user create command to create RADOS Gateway users.

When creating RADOS Gateway users, both the --uid and --display-name options are required, and specify a unique account name and a human friendly display name. Use the --access-key and --secret options to specify a custom AWS account and secret key for the RADOS user.

[ceph: root@node /]# radosgw-admin user create --uid=testuser \
  --display-name="Test User" --email=test@example.com \
  --access-key=12345 --secret=67890
...output omitted...
    "keys": [
        {
            "user": "testuser",
            "access_key": "12345",
            "secret_key": "67890"
        }
...output omitted...

If the access key and secret key are not specified, the radosgw-admin command automatically generates them and displays them in the output.

[ceph: root@node /]# radosgw-admin user create --uid=s3user \
  --display-name="Amazon S3 API user"
...output omitted...
    "keys": [
        {
            "user": "s3user",
            "access_key": "8PI2D9ARWNGJI99K8TOS",
            "secret_key": "brKaQdhyR022znVVVVdDLuAEafRjbrAorr0GoXN1"
        }
...output omitted...

To regenerate only the secret key of an existing user, use the radosgw-admin key create command with the --gen-secret option.

[ceph: root@node /]# radosgw-admin key create --uid=s3user \
  --access-key="8PI2D9ARWNGJI99K8TOS" --gen-secret
...output omitted...
    "keys": [
        {
            "user": "s3user",
            "access_key": "8PI2D9ARWNGJI99K8TOS",
            "secret_key": "MFVxrGNMBjKOO7JscLFbEyrEmJFnLl43PHSswpLC"
        }
...output omitted...

To add an access key to an existing user, use the --gen-access-key option. Creating additional keys is convenient for granting the same user access to multiple applications that require different or unique keys.

[ceph: root@node /]# radosgw-admin key create --uid=s3user --gen-access-key
{
...output omitted...
    "keys": [
        {
            "user": "s3user",
            "access_key": "8PI2D9ARWNGJI99K8TOS",
            "secret_key": "MFVxrGNMBjKOO7JscLFbEyrEmJFnLl43PHSswpLC"
        },
        {
            "user": "s3user",
            "access_key": "GPYJGPSONURDY7SG0LLO",
            "secret_key": "T7jcG5YgEqqPxWMkdCTBsY0DM3rgIOmqkmtjRlCX"
        }
...output omitted...

To remove an access key and related secret key from a user, use the radosgw-admin key rm command with the --access-key option. This is useful for removing single application access without impacting access with other keys.

[ceph: root@node /]# radosgw-admin key rm --uid=s3user \
  --access-key=8PI2D9ARWNGJI99K8TOS
{
    "keys": [
        {
            "user": "s3user",
            "access_key": "GPYJGPSONURDY7SG0LLO",
            "secret_key": "T7jcG5YgEqqPxWMkdCTBsY0DM3rgIOmqkmtjRlCX"
        }

Temporarily disable and enable RADOS Gateway users by using the radosgw-admin user suspend and radosgw-admin user enable commands. When suspended, a user's subusers are also suspended and unable to interact with the RADOS Gateway service.

You can modify user information such as email, display name, keys and access control level. The access control levels are: read, write, readwrite, and full. The full access level includes the readwrite level and the access control management capability.

[ceph: root@node /]# radosgw-admin user modify --uid=johndoe --access=full

To remove a user and also delete their objects and buckets, use the --purge-data option.

[ceph: root@node /]# radosgw-admin user rm --uid=s3user --purge-data

Set quotas to limit the amount of storage a user or bucket can consume. Set the quota parameters first, then enable the quota. To disable a quota, set a negative value for the quota parameter.

Bucket quotas apply to all buckets owned by a specific UUID, regardless of the user accessing or uploading to those buckets.

In this example, the quota for the app1 user is set to a maximum of 1024 objects. The user quota is then enabled.

[ceph: root@node /]# radosgw-admin quota set --quota-scope=user --uid=app1 \
  --max-objects=1024
[ceph: root@node /]# radosgw-admin quota enable --quota-scope=user --uid=app1

Similarly, apply quotas to buckets by setting the --quota-scope option to bucket. In this example, the loghistory bucket is set for a maximum size of 1024 bytes.

[ceph: root@node /]# radosgw-admin quota set --quota-scope=bucket \
--uid=loghistory --max-objects=1024B
[ceph: root@node /]# radosgw-admin quota enable --quota-scope=bucket --uid=loghistory

Global quotas affect all buckets in the cluster.

[ceph: root@node /]# radosgw-admin global quota set --quota-scope bucket \
--max-objects 2048
[ceph: root@node /]# radosgw-admin global quota enable --quota-scope bucket

To be implemented in a zone and period configuration, use the radosgw-admin period update --commit command to commit the change. Alternately, restart the RGW instances to implement the quota.

Use the following commands to retrieve user information and statistics:

Storage administrators monitor usage statistics to determine the storage consumption or user bandwidth usage. Monitoring can also help find inactive applications or inappropriate user quotas.

Use the radosgw-admin user stats and radosgw-admin user info commands to view user information and statistics.

[ceph: root@node /]# radosgw-admin user info --uid=uid
[ceph: root@node /]# radosgw-admin user stats --uid=uid

Use the radosgw-admin usage show command to show the usage statistics of a user between specific dates.

[ceph: root@node /]# radosgw-admin usage show --uid=uid  --start-date=start  --end-date=end

View the statistics for all users by using the radosgw-admin usage show command. Use these overall statistics to help understand object storage patterns and plan the deployment of new instances for scaling the RADOS gateway service.

[ceph: root@node /]# radosgw-admin usage show --show-log-entries=false

The Amazon S3 API supports several bucket URL formats, including either http://server.example.com/bucket/ or http://bucket.server.example.com/. Some clients, such as the s3cmd command, only support the second URL format. RADOS Gateway does not enable this format by default. To enable the second URL format, set the rgw_dns_name parameter for your DNS suffix.

[ceph: root@node /]# ceph config set client.rgw rgw_dns_name dns_suffix

where dns_suffix is the fully qualified domain name to be used to create your bucket's name.

In addition to configuring rgw_dns_name, you must configure your DNS server with a wildcard DNS record for that domain that points to the RADOS Gateway IP address. Syntax for implementing wildcard DNS entries varies for different DNS servers.

[ceph: root@node /]# aws s3 mb s3://demobucket

[ceph: root@node /]# aws --acl=public-read-write s3 cp /tmp/demoobject \
s3://demobucket/demoobject

{
  "Version": "2021-03-10",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": ["arn:aws:iam::testaccount:user/testuser"]
      },
      "Action": "s3:ListBucket",
      "Resource": [
      "arn:aws:s3:::testbucket"
      ]
    }
  ]
}