After completing this section, you should be able to describe, plan, and set up SAP HANA multitarget system replication on multiple sites and related use cases.
SAP HANA Multitarget Replication (MTR) is a specific setup of SAP HANA system replication. It supports simultaneous parallel copies from one source to multiple target databases. As described in more detail at https://help.sap.com/docs/SAP_HANA_PLATFORM/4e9b18c116aa42fc84c7dbfd02111aba/ba457510958241889a459e606bbcf3d3.html, in a multitarget system replication, primary and secondary systems can replicate data changes to more than one system.
This method increases the redundancy and reduces the risk of data loss.
The primary system must synchronize its changes to the multiple secondary systems, which slightly increases the load on the primary system.
In this setup, you must install an additional SAP HANA database with the same SID and InstanceNumber values as the primary side, and then register this database as secondary to the primary database.
If this secondary site is not in the same data center, then the async replication mode is recommended.
You can verify the currently running SAP HANA System Replication mode with the following command:
[sidadm]% hdbnsutil -sr_stateThe following parameters are required for this setup:
| Name | Description | Example |
|---|---|---|
| node_name | Name of the node | hana01, hana02, … |
| hana_site | Location name of the HANA instance | DC1, DC2, DC3 |
| node_role | Role of the node | primary or secondary/worker |
| replicationMode | Replication Modes for HANA System Replication, https://bit.ly/3QIvfxi |
sync or syncmem
|
| operationMode | Operation Mode for HANA System Replication, https://bit.ly/3QvWrja |
delta_datashipping, logreplay, or logreplay_readaccess
|
| sap_hana_sid | SID system identifier |
RH1
|
| sap_hana_instance_number | Instance number |
00
|
| PREFER_SITE_TAKEOVER | Should the resource agent prefer to switch over to the secondary instance instead of restarting the primary instance locally? |
true, false, or never
|
| DUPLICATE_PRIMARY_TIMEOUT | Time period before automatic failover is possible | Defaults to 7200 seconds |
| AUTOMATED_REGISTER | The cluster registers the former primary as secondary during failover |
false
|
| register_secondaries_on_takeover | Parameter in the SAP HANA global.ini file to re-register the secondary if the primary changed |
false
|
Legend: P = Primary, S = Secondary, F = Failed
| Use case | DC1 | DC2 | DC3 | Description |
|---|---|---|---|---|
| Normal | P1 | S1 | S1 | No changes |
| Failover P2 | F→S2 | P1 | S1→S2 |
register_secondaries_on_takeover
|
| Primary DC2 | S2 | P2 | S2 | Primary is running on DC2 |
| Failover P3 | S2→S3 | P2→S3 | S2→P3 |
register_secondaries_on_takeover
|
To re-establish the connection of the third secondary site to the new primary site, you must set the register_secondaries_on_takeover parameter in the SAP HANA global.ini file.
Pacemaker manages the failover and high availability of the first two sites, by using the resource agents.
Based on the previously described installation, these additional installation steps are necessary:
SAP HANA installation on a third site.
Register the third site as a secondary SAP HANA replication server.
Add the nodes of the third site to the cluster.
Create constraints to avoid SAP resource agent resources running on nodes of the third site.
Avoid SAP HANA topology running in node 2 of the DC3 site by setting up constraints:
[root]#pcs constraint location rsc_SAPHanaTopology_RH1_HDB00-clone \>avoids dc3hana02
The content of the /etc/sudoers.d/20-saphana file should be as follows:
# SAPHanaSR-ScaleOut needs for srHook Cmnd_Alias SOK = /usr/sbin/crm_attribute -n hana_rh1_glob_srHook -v SOK -t crm_config -s SAPHanaSR Cmnd_Alias SFAIL = /usr/sbin/crm_attribute -n hana_rh1_glob_srHook -v SFAIL -t crm_config -s SAPHanaSR rh1adm ALL=(ALL) NOPASSWD: SOK, SFAIL Defaults!SOK, SFAIL !requiretty
Start the SAP HANA database on all the data centers:
[rh1adm]# sapcontrol -nr 00 -function StartSystemConfigure general cluster properties:
[root]#pcs resource defaults update resource-stickiness=1000[root]#pcs resource defaults update migration-threshold=5000
Create the SAPHanaTopology resource:
[root]#pcs resource create rsc_SAPHanaTopology_RH1_HDB00 SAPHanaTopology \>SID=RH1 InstanceNumber=00 op methods interval=0s timeout=5 op monitor \>interval=10 timeout=600[root]#pcs resource clone rsc_SAPHanaTopology_RH1_HDB00 clone-node-max=1 \>interleave=true
Create the SAPHanaController resource:
[root]#pcs resource create rsc_SAPHana_RH1_HDB00 SAPHanaController SID=RH1 \>InstanceNumber=00 PREFER_SITE_TAKEOVER=true DUPLICATE_PRIMARY_TIMEOUT=7200 \>AUTOMATED_REGISTER=true op demote interval=0s timeout=320 op methods \>interval=0s timeout=5 op monitor interval=59 role="Master" timeout=700 \>op monitor interval=61 role="Slave" timeout=700 op promote interval=0 \>timeout=3600 op start interval=0 timeout=3600 op stop interval=0 timeout=3600[root]#pcs resource promotable rsc_SAPHana_RH1_HDB00 promoted-max=1 \>clone-node-max=1 interleave=true
Create a virtual IP address resource:
[root]# pcs resource create rsc_ip_SAPHana_RH1_HDB00 IPaddr2 ip="192.168.0.15"Create constraints to start SAPHanaTopology before SAPHanaController:
[root]#pcs constraint order SAPHanaTopology_RH1_00-clone then \>SAPHana_RH1_00-master symmetrical=false
Create a constraint to ensure the IPaddr2 resource with master SAPHanaController:
[root]#pcs constraint colocation add vip_RH1_00 with master \>SAPHana_RH1_00-master 2000
Create a constraint to avoid starting SAPHanaTopology on majoritymaker:
[root]#pcs constraint location rsc_SAPHanaTopology_RH1_HDB00-clone \>avoids majoritymaker
Colocate the IPaddr2 resource with the master SAPHana resource:
[root]#pcs constraint colocation add rsc_ip_SAPHana_RH1_HDB00 \>with master rsc_SAPHana_RH1_HDB00-clone
List all SAP HANA database instances:
[rh1adm]# /usr/sap/hostctrl/exe/sapcontrol -nr 00 -function GetSystemInstanceList
10.04.2019 08:38:21
GetSystemInstanceList
OK
hostname, instanceNr, httpPort, httpsPort, startPriority, features,
dispstatus
dc1hana01, 00, 50013, 50014, 0.3, HDB|HDB_WORKER, GREEN
dc1hana03, 00, 50013, 50014, 0.3, HDB|HDB_STANDBY, GREEN
dc1hana02, 00, 50013, 50014, 0.3, HDB|HDB_WORKER, GREEN
dc1hana04, 00, 50013, 50014, 0.3, HDB|HDB_WORKER, GREENVerify the landscape host configuration:
[rh1adm]# HDBSettings.sh landscapeHostConfiguration.py
ok
rh1adm@dc1hana01:/usr/sap/RH1/HDB00> HDBSettings.sh
landscapeHostConfiguration.py
| Host | Host | Host | Failover | Remove | Storage |
Storage | Failover | Failover | NameServer | NameServer |
IndexServer | IndexServer | Host | Host | Worker | Worker |
| | Active | Status | Status | Status | Config |
Actual | Config | Actual | Config | Actual | Config
| Actual | Config | Actual | Config | Actual |
| | | | | | Partition |
Partition | Group | Group | Role | Role | Role
| Role | Roles | Roles | Groups | Groups |
| --------- | ------ | ------ | -------- | ------ | --------- |
--------- | -------- | -------- | ---------- | ---------- |
----------- | ----------- | ------- | ------- | ------- | ------- |
| dc1hana01 | yes | ok | | | 1 |
1 | default | default | master 1 | master | worker |
master | worker | worker | default | default |
| dc1hana02 | yes | ok | | | 2 |
2 | default | default | master 3 | slave | worker |
slave | worker | worker | default | default |
| dc1hana03 | yes | ok | | | 2 |
2 | default | default | master 3 | slave | worker |
slave | worker | worker | default | default |
| dc1hana04 | yes | ignore | | | 0 |
0 | default | default | master 2 | slave | standby |
standby | standby | standby | default | - |Verify HANA system replication:
[rh1adm]# python /usr/sap/RH1/HDB02/exe/python_support/systemReplicationStatus.py
| Host | Port | Service Name | Volume ID | Site ID | Site Name | Secondary |
| | | | | | | Host |
| ----- | ----- | ------------ | --------- | ------- | --------- | --------- |
| node1 | 30201 | nameserver | 1 | 1 | DC1 | node2 |
| node1 | 30207 | xsengine | 2 | 1 | DC1 | node2 |
| node1 | 30203 | indexserver | 3 | 1 | DC1 | node2 |
Secondary | Secondary | Secondary | Secondary | Replication | Replication |
Port | Site ID | Site Name | Active Status | Mode | Status |
--------- | --------- | --------- | ------------- | ----------- | ----------- |
30201 | 2 | DC2 | YES | SYNCMEM | ACTIVE |
30207 | 2 | DC2 | YES | SYNCMEM | ACTIVE |
30203 | 2 | DC2 | YES | SYNCMEM | ACTIVE |
status system replication site "2": ACTIVE
overall system replication status: ACTIVE
Local System Replication State
~~~~~~~~~~
mode: PRIMARY
site id: 1
site name: DC1Test SrConnectionChangedHook:
# To check if hook scripts are working [rh1adm]#cdtrace[rh1adm]#awk '/ha_dr_SAPHanaSR.*crm_attribute/ \>{ printf "%s %s %s %s\n",$2,$3,$5,$16 }' nameserver_*2018-05-04 12:34:04.476445 ha_dr_SAPHanaSR SFAIL 2018-05-04 12:53:06.316973 ha_dr_SAPHanaSR SOK [rh1adm]#grep ha_dr_ *
Verify the SAPHanaTopology resource:
[rh1adm]# pcs resource show SAPHanaTopology_RH1_00-clone
Clone: SAPHanaTopology_RH1_00-clone
Meta Attrs: clone-max=2 clone-node-max=1 interleave=true
Resource: SAPHanaTopology_RH1_00 (class=ocf provider=heartbeat type=SAPHanaTopology)
Attributes: SID=RH1 InstanceNumber=00
Operations: start interval=0s timeout=600 (SAPHanaTopology_RH1_00-start-interval-0s)
stop interval=0s timeout=300 (SAPHanaTopology_RH1_00-stop-interval-0s)
monitor interval=10 timeout=600 (SAPHanaTopology_RH1_00-monitor-interval-10s)Verify the SAPHana resource:
[root]# pcs resource config SAPHana_RH1_00
Clone: SAPHana_RH1_00-clone
Meta Attrs: clone-max=2 clone-node-max=1 interleave=true notify=true promotable=true
Resource: SAPHana_RH1_00 (class=ocf provider=heartbeat type=SAPHana)
Attributes: AUTOMATED_REGISTER=true DUPLICATE_PRIMARY_TIMEOUT=180 InstanceNumber=00 PREFER_SITE_TAKEOVER=true SID=RH1
Operations: demote interval=0s timeout=3600 (SAPHana_RH1_00-demote-interval-0s)
methods interval=0s timeout=5 (SAPHana_RH1_00-methods-interval-0s)
monitor interval=61 role=Slave timeout=700 (SAPHana_RH1_00-monitor-interval-61)
monitor interval=59 role=Master timeout=700 (SAPHana_RH1_00-monitor-interval-59)
promote interval=0s timeout=3600 (SAPHana_RH1_00-promote-interval-0s)
start interval=0s timeout=3600 (SAPHana_RH1_00-start-interval-0s)
stop interval=0s timeout=3600 (SAPHana_RH1_00-stop-interval-0s)Verify the cluster:
[root]# pcs status --full
Cluster name: hanascaleoutsr
Stack: corosync
Current DC: majoritymaker (9) (version 1.1.18-11.el7_5.4-2b07d5c5a9)
- partition with quorum
Last updated: Tue Mar 26 16:34:22 2019
Last change: Tue Mar 26 16:34:03 2019 by root via crm_attribute on
dc2hana01
9 nodes configured
20 resources configured
Online: [ dc1hana01 (1) dc1hana02 (2) dc1hana03 (3) dc1hana04 (4)
dc2hana01 (5) dc2hana02 (6) dc2hana03 (7) dc2hana04 (8) majoritymaker
(9) ]
......
--------------------------------------------------------
1 PROMOTED master1:master:worker:master 150 DC1
2 DEMOTED master2:slave:worker:slave 110 DC1
3 DEMOTED slave:slave:worker:slave -10000 DC1
4 DEMOTED master3:slave:standby:standby 115 DC1
5 DEMOTED master2:master:worker:master 100 DC2
6 DEMOTED master3:slave:worker:slave 80 DC2
7 DEMOTED slave:slave:worker:slave -12200 DC2
8 DEMOTED master1:slave:standby:standby 80 DC2
9 :shtdown:shtdown:shtdownVerify the VIP1 status:
[root]# pcs resource show vip_RH1_00
Resource: vip_RH1_00 (class=ocf provider=heartbeat type=IPaddr2)
Attributes: ip=192.168.0.15
Operations: start interval=0s timeout=20s (vip_RH1_00-start-interval-0s)
stop interval=0s timeout=20s (vip_RH1_00-stop-interval-0s)
monitor interval=10s timeout=20s (vip_RH1_00-monitor-interval-10s)This concludes the section for SAP HANA scale-out multitarget system replication.