HPE 3PAR Simulator

[Mayumi]

Hi all,
Could you tell me about the function of 3PAR Simulator.
I tried to auto failover via 2 3PAR simulators and Quorum witness. But it doesn't work.
How setup Quorum Witness and run Witness using 3PAR Simulator?

[sbhat09]

Hi Mayumi,

What is the version of QW you installed?

Is the QW server pingable from both the simulators?

Regards,

Srinivas

[Mayumi]

I installed QW using rcopy_qw-2.0.001.HyperV from 3PAR Quorum Witness Hyper_V_2.0.001_QR482-11024.

QW server is pingable from both the simulators.

[sbhat09]

Hi Mayumi,

What is the error you are getting?

What are the steps by step procedure you tried to do failover? And at what stage it doesn't work as expected?

Also please provide below CLI command outputs from both the simulators:

showrcopy –d

showrcopy -qw targets

Regards,

Srinivas

[Mayumi]

Hi Srinivas,

The following was done in order to work automatic failover.
(I referred to the demo of this video. https://www.youtube.com/watch?v=Eet92dOra24 )

1. Setted the automatic recovery policy and automatic failover policy with the following command:
3par_sim0 cli%  setrcopygroup pol auto_recover <group_name>
3par_sim0 cli%  setrcopygroup pol auto_failover <group_name>

2. RCIP port downed.
3par_sim0 cli%  controlport rcip state down -f 0:3:1 1:3:1

3. Stopped command response from 3par sim0 IP on QW.
roo@localhost ~#  iptables -A OUTPUT -d 192.168.20.180 -j DROP

　　→I want to start automatic failover at this point, but it doesn't work.

How setup in order to start auto failover ?

<Command result>
・3par_sim0 cli% showrcopy -d

Remote Copy System Information
Status: Started, Normal

Target Information

Name        ID Type  Status  Options Version  Policy                Socks       Buffsz   SysName  SysID  RemoteName    RemoteID
3par_sim1  2   IP       failed   -                     26   mirror_config         20   2097152   None         NA      None                  NA

Link Information

Target          Name                      IPC        ID   Node   Address      Status  Options
3par_sim1     3par_sim1_0_3_1    RCs05    5    0:3:1     192.1.82.77   Down    -
3par_sim1     3par_sim1_1_3_1     RCs16     6    1:3:1      192.1.82.78   Down   -
receive          rcv_ip_031              RCr07    7    0:3:1     receive         Up        -
receive          rcv_ip_131               RCr18     8    1:3:1     receive         Up        -

Group Information

Name          ID  Target        Status      Role         Mode   LocalUserCpg LocalSnapCpg RmUserCpg RmSnapCpg Options
rc_group1    5    3par_sim1  Stopped   Primary   Sync     auto_recover,auto_failover
LocalVV     ID   RemoteVV   ID  SyncStatus  Resync_ss   Sync_ss VV_iter R_iter S_iter LastSyncTime
vv-1             7    rvv-1              7    Stopped      rcpy.5.7.2      none          NA     NA    NA    NA

・3par_sim1 cli% showrcopy -d

Remote Copy System Information
Status: Started, Normal

Target Information

Name ID Type Status Options Version Policy Socks Buffsz SysName SysID RemoteName RemoteID
3par_sim0 2 IP failed - 26 mirror_config 20 2097152 None NA None NA

Link Information

Target Name IPC ID Node Address Status Options
3par_sim0 3par_sim0_0_3_1 RCs05 5 0:3:1 192.1.82.75 Down -
3par_sim0 3par_sim0_1_3_1 RCs16 6 1:3:1 192.1.82.76 Down -
receive rcv_ip_031 RCr07 7 0:3:1 receive Up -
receive rcv_ip_131 RCr18 8 1:3:1 receive Up -

Group Information

Name ID Target Status Role Mode LocalUserCpg LocalSnapCpg RmUserCpg RmSnapCpg Options
rc_group1.r99678 5 3par_sim0 Stopped Secondary Sync auto_recover,auto_failover
LocalVV ID RemoteVV ID SyncStatus Resync_ss Sync_ss VV_iter R_iter S_iter LastSyncTime
rvv-1 7 vv-1 7 Stopped none none NA NA NA

・3par_sim0 cli% showrcopy -qw targets

Remote Copy System Information
Status: Started, Normal

Target Information

Name ID Type Status Policy QW-IP Q-Status Q-Status-Qual
3par_sim1 2 IP failed mirror_config 192.168.20.185 Re-starting Target not ready

・3par_sim1 cli% showrcopy -qw targets

Remote Copy System Information
Status: Started, Normal

Target Information

Name ID Type Status Policy QW-IP Q-Status Q-Status-Qual
3par_sim0 2 IP failed mirror_config 192.168.20.185 Re-starting Target not ready

[sbhat09]

Hi Mayumi,

Thanks for the details.

But I am yet to understand: At what stage of the failover it doesn't work as expected? And what is the error you are getting?

Also the logs you sent are after the failover attempt. There the rcopy links are down, groups are stopped and QW targets are not ready.

I would need the logs prior to failover attempt (expecting a  working Remote copy set up with working QW).

Regards,

Srinivas

[Mayumi]

Hi Srinivas,

Two pairs of 3par simulator were in a ESXi host and witness were build in a Hyper-V.

[primary 3par (3par_sim0)]===①X====[secondary 3par (3par_sim1)]

               ||                                                                              ||

          ②X                                                                            ||

               ||                                                                              ||

               ||                                                                              ||

               ========[Quorum Witness]==========

I expected that the Secondary 3PAR (3par_sim1)  become primary when I disconnected both ① and ② manually.

No error is displayed. However, automatic failover does not start on the Secondary.

<the logs prior to failover attempt>

3par_sim0 cli% showrcopy -d

Remote Copy System Information
Status: Started, Normal

Target Information

Name ID Type Status Options Version Policy Socks Buffsz SysName SysID RemoteName RemoteID
3par_sim1 2 IP ready - 26 mirror_config 20 2097152 3par_sim1 99679 3par_sim0 2

Link Information

Target Name IPC ID Node Address Status Options
3par_sim1 3par_sim1_0_3_1 RCs05 5 0:3:1 192.1.82.77 Up -
3par_sim1 3par_sim1_1_3_1 RCs16 6 1:3:1 192.1.82.78 Up -
receive rcv_ip_031 RCr07 7 0:3:1 receive Up -
receive rcv_ip_131 RCr18 8 1:3:1 receive Up -

Group Information

Name ID Target Status Role Mode LocalUserCpg LocalSnapCpg RmUserCpg RmSnapCpg Options
rc_group1 6 3par_sim1 Started Primary Sync auto_recover,auto_failover
LocalVV ID RemoteVV ID SyncStatus Resync_ss Sync_ss VV_iter R_iter S_iter LastSyncTime
vv-1 7 rvv-1 7 Synced none none NA NA NA NA

3par_sim0 cli%
3par_sim0 cli%
3par_sim0 cli% showrcopy -qw targets

Remote Copy System Information
Status: Started, Normal

Target Information

Name ID Type Status Policy QW-IP Q-Status Q-Status-Qual
3par_sim1 2 IP ready mirror_config 192.168.20.185 Started

3par_sim1 cli% showrcopy -d

Remote Copy System Information
Status: Started, Normal

Target Information

Name ID Type Status Options Version Policy Socks Buffsz SysName SysID RemoteName RemoteID
3par_sim0 2 IP ready - 26 mirror_config 20 2097152 3par_sim0 99678 3par_sim1 2

Link Information

Target Name IPC ID Node Address Status Options
3par_sim0 3par_sim0_0_3_1 RCs05 5 0:3:1 192.1.82.75 Up -
3par_sim0 3par_sim0_1_3_1 RCs16 6 1:3:1 192.1.82.76 Up -
receive rcv_ip_031 RCr07 7 0:3:1 receive Up -
receive rcv_ip_131 RCr18 8 1:3:1 receive Up -

Group Information

Name ID Target Status Role Mode LocalUserCpg LocalSnapCpg RmUserCpg RmSnapCpg Options
rc_group1.r99678 6 3par_sim0 Started Secondary Sync auto_recover,auto_failover
LocalVV ID RemoteVV ID SyncStatus Resync_ss Sync_ss VV_iter R_iter S_iter LastSyncTime
rvv-1 7 vv-1 7 Synced none none NA NA NA NA

3par_sim1 cli%
3par_sim1 cli%
3par_sim1 cli% showrcopy -qw targets

Remote Copy System Information
Status: Started, Normal

Target Information

Name ID Type Status Policy QW-IP Q-Status Q-Status-Qual
3par_sim0 2 IP ready mirror_config 192.168.20.185 Started

[sbhat09]

Hi Mayumi,

Thanks for the details. Present remote copy set up with peer persistence looks fine.

Before trying the failover (ATF - Automatic Transparent Failover) by disconnecting both ① and ②, please try the below manual failover and failback procedure and check you are getting the expected results. After every DR command you run, please capture 'showrcopy -d' output from both simulators.

Please refer the attached image.

If everything goes as expected, then you can try to disconnect both ① and ② as you did before. Both links have to go down almost immediately. And then observe the group and volume status. I think the primary group will remain primary and volumes will be failsafe. Secondary group will be primary-rev and volumes will be active.

Please let me know the results after you complete it.

Regards,

Srinivas

[Mayumi]

Hi Srinivas, 

Before trying the failover by disconnecting both ① and ②,  tried the manual failover and failback procedure and I could getting the expected results.

Everything went as expected, so I tried to disconnect both ① and ② as I did before.
Both links went down almost immediately, so I could getting the expected work.

Until then, it was failing because it was executing in sequence.

< 'showrcopy -d' output from both simulators>

3par_sim0 cli% showrcopy -d

Remote Copy System Information
Status: Started, Normal

Target Information

Name ID Type Status Options Version Policy Socks Buffsz SysName SysID RemoteName RemoteID
3par_sim1 2 IP ready - 26 mirror_config 20 2097152 3par_sim1 99679 3par_sim0 2

Link Information

Target Name IPC ID Node Address Status Options
3par_sim1 3par_sim1_0_3_1 RCs05 5 0:3:1 192.1.82.77 Up -
3par_sim1 3par_sim1_1_3_1 RCs16 6 1:3:1 192.1.82.78 Up -
receive rcv_ip_031 RCr07 7 0:3:1 receive Up -
receive rcv_ip_131 RCr18 8 1:3:1 receive Up -

Group Information

Name ID Target Status Role Mode LocalUserCpg LocalSnapCpg RmUserCpg RmSnapCpg Options
rc_group1 6 3par_sim1 Started Secondary Sync auto_recover,auto_failover,auto_synchronize
LocalVV ID RemoteVV ID SyncStatus Resync_ss Sync_ss VV_iter R_iter S_iter LastSyncTime
vv-1 7 rvv-1 7 Synced none none NA NA NA NA

3par_sim1 cli% showrcopy -d

Remote Copy System Information
Status: Started, Normal

Target Information

Name ID Type Status Options Version Policy Socks Buffsz SysName SysID RemoteName RemoteID
3par_sim0 2 IP ready - 26 mirror_config 20 2097152 3par_sim0 99678 3par_sim1 2

Link Information

Target Name IPC ID Node Address Status Options
3par_sim0 3par_sim0_0_3_1 RCs05 5 0:3:1 192.1.82.75 Up -
3par_sim0 3par_sim0_1_3_1 RCs16 6 1:3:1 192.1.82.76 Up -
receive rcv_ip_031 RCr07 7 0:3:1 receive Up -
receive rcv_ip_131 RCr18 8 1:3:1 receive Up -

Group Information

Name ID Target Status Role Mode LocalUserCpg LocalSnapCpg RmUserCpg RmSnapCpg Options
rc_group1.r99678 6 3par_sim0 Started Primary Sync auto_recover,auto_failover,auto_synchronize
LocalVV ID RemoteVV ID SyncStatus Resync_ss Sync_ss VV_iter R_iter S_iter LastSyncTime
rvv-1 7 vv-1 7 Synced none none NA NA NA NA

Thank you for your support !!!

Regards,

Mayumi

[sbhat09]

Hi Mayumi,

Great to hear that from you.

Regards,

Srinivas

If you feel this was helpful please click the KUDOS! thumb below!

If that resolves your issue, please mark it as ACCEPTED SOLUTION!