Red Hat High Availability Clustering for SAP Solutions

Figure 0.1: Classroom environment

In this course, the main computer system that is used for hands-on learning activities is workstation. The workstation virtual machine (VM) is the only one with a graphical desktop. Always log in directly to workstation first.

From workstation, use SSH for command-line access to all other VMs. The workstation machine has a standard user account, student, with student as the password. The student user account can become the root user if necessary. No exercise in this course requires you to log in directly as root, but if you must, then the password is redhat.

You perform all exercises on the hana1, hana2, nodea, nodeb, nodec, and noded machines. For these machines, the password for the student user is student, with the same privileges as on the workstation machine.

As you can see in SectionFigure 0.1: Classroom environment, all VMs share the lab.example.com DNS domain, 172.25.250.0/24.

Alongside the lab.example.com network, three other networks are also in use: private.example.com (192.168.0.0/24) for private cluster communications, and san01.example.com (192.168.1.0/24) and san02.example.com (192.168.2.0/24) for iSCSI and NFS storage traffic.

In the classroom environment, only the private.example.com network is used for private cluster communications. The private cluster communication network is critical in the cluster infrastructure, because the whole cluster requires this network to work. For this reason, Red Hat recommends to improve the cluster resilience by using network redundancy in production environments. For more details, see the RH436 course, Red Hat High Availability Clustering.

The system named bastion must always be running. The bastion system acts as a router between the network that connects your lab machines and the classroom network. If bastion is down, then other lab machines might not function properly, or might even hang during boot.

Several systems in the classroom provide supporting services. The utility.example.com server acts as an NFS file server for the SAP software.

An iSCSI and NFS storage server, storage.lab.example.com, is also provided. Information about how to use these servers is provided in the instructions for those activities.

The Classroom Machines table provides information for the different machines that are used in the classroom environment, including their role and IP addresses.

Fencing Environment

Fencing is one important part of a high availability cluster. It restricts the access of an unresponsive node to the cluster resources. Fencing is explained in detail in the RH436 course, Red Hat High Availability Clustering. Two different fencing methods are used during this course:

Fencing through simulated BMC

The following machines are connected to the fence network.

Figure 0.2: Simulated Baseboard Management Controller (BMC) environment

Because virtual machines have no embedded BMC device for power management, the BMC function is simulated by using the power machine.

The simulated BMC mechanism performs monitoring and management tasks remotely on the cluster nodes.

The IP addresses of the BMC devices (192.168.0.101, 192.168.0.102, 192.168.0.103, 192.168.0.104, 192.168.0.105, and 192.168.0.106 for the nodea, nodeb, nodec, noded, hana1, and hana2 machines, respectively) are hosted on the power machine.

The BMC IP addresses and node names are assigned when the classroom environment is built. The openstackbmc service runs on the power machine with one process for each power-managed cluster node. This service responds to the Intelligent Platform Management Interface (IPMI) requests on behalf of the corresponding node.

[root@power ~]# systemctl status openstackbmc
  openstackbmc.service - OpenStack BMC using fakeipmi
   Loaded: loaded (/etc/systeml/system/openstackbmc.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2021-05-18 03:01:24 EDT; 2h 21m ago
  Process: 912 ExecStart=/usr/local/bin/openstackbme-wrap.bash (code=exited, status=0/SUCCESS)
    Tasks: 13 (limit: 5036)
   Memory: 171.8M
   CGroup: /system.slice/openstackbmc.service
           |-1285 python /usr/local/bin/openstackbmc.py --project-name=ole-fb6154cc-2848-48b8-b81b-ac78d341a3cb --vm-name=noded
           |-1288 python /usr/local/bin/openstackbmc.py --project-name=ole-fb6154cc-2848-48b8-b81b-ac78d341a3cb --vm-name=nodec
           |-1291 python /usr/local/bin/openstackbmc.py --project-name=ole-fb6154cc-2848-48b8-b81b-ac78d341a3cb --vm-name=nodeb
           |-1294 python /usr/local/bin/openstackbmc.py --project-name=ole-fb6154cc-2848-48b8-b81b-ac78d341a3cb --vm-name=nodea

...output omitted...

For all the BMC devices, the login is admin and the password is password.

Fencing through simulated chassis

The fence network here is similar to the previous one.

Figure 0.3: Simulated chassis

The second fencing method that is used in this course simulates a management chassis, such as ibmblade, hpblade, or bladecenter).

With this method, only the chassis IP address is needed to request fencing.

This fencing method uses the fence_ipmilan custom script and the pcmk_host_map parameter to assign a plug number to each cluster node. A request that is sent to the chassis IP (192.168.0.100) includes the node to fence, which the plug number specifies. The fence_ipmilan script converts the request to an IPMI call to the Fencing through simulated BMC method.

[root@nodeX ~]# cat /usr/sbin/fence_ipmilan
...output omitted...
ip_name_mapping = {
    "nodea": socket.gethostbyname("bmc-nodea"),
    "nodeb": socket.gethostbyname("bmc-nodeb"),
    "nodec": socket.gethostbyname("bmc-nodec"),
    "noded": socket.gethostbyname("bmc-noded"),
}
...output omitted...

In this classroom, the power machine, to simulate a chassis, performs the request, such as to power off the virtual machine that is associated with the node to fence.

Procedures for managing high availability clusters in the classroom differ from production environments. Typically, high availability clusters require resilient, low-latency communication between all nodes. Nodes can be taken offline or be rebooted for maintenance, without functional loss. In production, the full environment is rarely if ever shut down completely.

The major difference in the classroom environment is that it uses virtual machines that are either deployed online or on a single physical system, whereas production environments typically use bare metal systems. Most training locations, both online and physical, automatically shut down student systems after timed use or after class hours.

Although automated shutdowns are not graceful, current high availability clusters are resilient enough for classrooms to restart and operate without difficulty.

Typically, SAP HANA and SAP S/4HANA require much larger systems and dedicated storage in production. SAP HANA has dedicated disk performance requirements, so that dedicated striped data volumes are required in production environments. This course uses one virtualized disk and reduced server memory to minimize the resource consumption in the environment, to increase the number of students who can take this training in parallel. Hence, the course uses SAP S/4HANA Foundation, which installs only certain core components of SAP S/4HANA, and not the full SAP S/4HANA Business Software, on the SAP HANA database. The main difference from a regular installation is the much smaller and faster database load. From an OS and installation procedure viewpoint, no difference applies.

A high availability cluster infrastructure is designed for long-term operation.

However, you might need to reboot nodes in the cluster. If you need to reboot all the nodes, then it is advisable to reboot each node individually one after another, because rebooting all nodes at the same time can cause service downtime.

Recommended classroom practice is to gracefully stop your high availability cluster when it is finished for an extended time. To stop the cluster services, execute the following command:

[root@node ~]# pcs cluster stop --all

If you forget to shut down manually, then your course environment might time out and be shut down automatically. If you encounter major issues with restarting your environment, then discard and reprovision your student environment by using the correct procedure for the delivery environment.

Resetting your classroom environment is the procedure to revert some or all of your classroom nodes to their original state when the course was first created. By resetting, you can clean your virtual machines and start the exercises again. This way, you can also clear any classroom issue that is blocking your progress and that is not getting solved.

This classroom has specific constraints for resetting part or all of your environment.

In most Red Hat training courses, individual systems can be reset separately as needed. However, in this course, resetting only a single cluster node results in that node losing necessary information and failing to communicate with other nodes as part of the cluster after resetting it.

Thus, the correct procedure is to remove first the node from your cluster and then to reset it. Removing a cluster node from an existing cluster is a two-step process.

First, remove the node from the cluster:

[root@oldnode ~]# pcs cluster remove newnode.example.com
...output omitted...

Then, adjust the fencing configuration, either by removing the dedicated fence device, or by reconfiguring the shared fence device to reflect the removal of one node:

[root@oldnode ~]# pcs stonith delete fence_deletednode
...output omitted...

If you need to reset any node in the course, then you must run all the lab exercises on that node before continuing.

The commands for resetting individual nodes are discussed in the upcoming "Controlling Your Systems" section.

[root@node ~]# pcs resource cleanup my_resource

You are assigned remote computers in a Red Hat Online Learning classroom. They are accessed through a web application hosted at . You should log in to this site using your Red Hat Customer Portal user credentials.

Controlling the Virtual Machines

The virtual machines in your classroom environment are controlled through a web page. The state of each virtual machine in the classroom is displayed on the page under the Online Lab tab.

Depending on the state of a machine, a selection of the following actions is available.

At the start of an exercise, if instructed to reset a single virtual machine node, click ACTION → Reset for only the specific virtual machine.

At the start of an exercise, if instructed to reset all virtual machines, click ACTION → Reset

If you want to return the classroom environment to its original state at the start of the course, you can click DELETE LAB to remove the entire classroom environment. After the lab has been deleted, you can click PROVISION LAB to provision a new set of classroom systems.

The Autostop Timer

The Red Hat Online Learning enrollment entitles you to a certain amount of computer time. To help conserve allotted computer time, the ROL classroom has an associated countdown timer, which shuts down the classroom environment when the timer expires.

To adjust the timer, click MODIFY to display the New Autostop Time dialog box. Set the number of hours until the classroom should automatically stop. Note that there is a maximum time of ten hours. Click ADJUST TIME to apply this change to the timer settings.