topic Re: P4500 Multi Site on same subnet? in StoreVirtual Storage

P4500 Multi Site on same subnet?

Peter J West — Tue, 19 Jul 2011 08:13:36 GMT

Hi,

We currently have 3 x P4500 G2 Storage Nodes which are configured as a single cluster. The IP's used are in a full class-c range and these are only used for storage traffic.

We have the option of installing another three nodes at a second location - this location will be reached using 2 x single mode fibre connections which will link the 2 storage switches at each location (so we have 4 storage switches in total - 2 at each site).

Because of this we don't really need to segregate the network into seperate subnets - but if we don't then will we still be able to use the multisite feature or will it cause issues?

In terms of configuring the system as multi-site is it really as simple as adding the new storage nodes into the management group, assigning them to a seperate site and then adding them to the existing cluster?

Thanks in advance for your help - so far i've been very impressed with the P4000-series.

Regards

Pete

Re: P4500 Multi Site on same subnet?

Bart_Heungens — Tue, 19 Jul 2011 08:28:42 GMT

Hi,

For a multi-site setup, the nodes can be in the same IP range.

However take some things into count...

What is the speed of the network between the 2 sites? There is a rule to take 50MBps per node in each site... Further the latency shouldn't be higher than 2ms...

Check also that all nodes are running at the same level of software (not mandatory but advised, you never know)

If all this is OK, you can add the new nodes in the management group. Create 2 sites and assign the nodes to the correct site... Finally add them to the cluster... At that moment the volumes will start to re-syncing across all nodes...

Kr,

Bart

Re: P4500 Multi Site on same subnet?

Jitun — Tue, 19 Jul 2011 09:19:49 GMT

Yes it is as simple as that.. Only thing to remember is how the Nodes are arranged in the Cluster.

When you are creating Network RAID 10 volumes.. Data is mirrored to adjacent Nodes.

Hence in a 6 Node cluster you will have all your Odd numberd Nodes in 1 Site and all Even Numbered Nodes in another Site.

Hence Data in Node 1 (Site 1) is mirrored to Data in Node 2 (Site 2)

So number the Nodes Logically and make sure the order of Nodes in Cluster is Correct.

Apart from that as Bart menitoned, ping times should be less than 2ms.

Re: P4500 Multi Site on same subnet?

Peter J West — Tue, 19 Jul 2011 11:06:41 GMT

Thanks for the answers guys - but the last comment about numbering has given me some food for thought and i'd just like to run through the setup.

Firstly we have an existing cluster which has three nodes in it - these are numbered node-1, node-2 and node-3. They are running Network Raid 10 which I believe means that each piece of data written to 1 node is stored on at least one other node, so in the event of a single node failure the system keeps running. All of the above nodes (1 to 3) are listed as being in Site A.

I've now installed three new nodes called node-4, node-5 and node-6. These have all been updated to the latest software and are sat on the end of 2 x 1gbp/s fibre links. We have two switches at Site B and we connect each storage node so that one interface goes to each switch - this is because we're running two SAN switches and hence that's why we have 2 fibre links (for redundancy). Nodes 4, 5 and 6 are all listed as being in Site B.

Do I need to change anything with the existing cluster or will it work if I just add the new nodes (4, 5 and 6) into the cluster? Will the data be replicated in the correct way or do I need to make some ajustments to my config to make the setup mirror correctly - and if so can someone please give me ideas on the procedure for this?

Many thanks for your help so far - given that the existing cluster has live data on it you can appreciate I want to exercise a little caution whilst establishing the mirror at Site B. :)

Pete

Re: P4500 Multi Site on same subnet?

Bart_Heungens — Tue, 19 Jul 2011 11:16:09 GMT

Don't worry too much about it. The multi site cluster will handle the copy of all blocks of data across the 2 sites...

In case that when you would have had 6 nodes in 1 site active, and then want to spread them across 2 sites, indeed you must take node 2, 4 and 6 and send them to the other site. This to avoid re-syncing of data.

Now with only 3 nodes, there will be alwayds re-levelling of the data...

If nodes 4, 5 and 6 are in Site B, add them to the cluster and you will see that all the data will be spread across the 6 nodes across 2 sites...

Kr,

Bart

Re: P4500 Multi Site on same subnet?

Peter J West — Tue, 19 Jul 2011 11:35:18 GMT

Thanks Bart,

Indeed i've gone ahead and done that and it's interesting to see how the UI handles the numbering.

It's now listing all 6 nodes but they're listed in the order 1,4,2,5,3,6. I guess this maps to how the mirroring works on adjacent nodes? So now the data from Node 1 is going to Node 4, 2 to 5 and 3 to 6. So effectively there is no data mirrored on Site A, but simply a full replica held at Site B?

So - one final question.

If all nodes at Site B go offline for a short period then the system will still work because all nodes at Site A are operative - but we will, of course, be vulnerable as failure of a single additional node at that point would be catastrophic. When the nodes at Site B come back online does it simply pick up where it left off and write any data that might be waiting to be mirrored?

Best regards

Pete

Re: P4500 Multi Site on same subnet?

Bart_Heungens — Tue, 19 Jul 2011 12:09:24 GMT

Hi,

First: 1 is not exactly a copy of 2, etc... It is more a sequential fill-up of data blocks (A, B, C, D, ...) There is some cross in the replication picture... In fact this is not that important but is is always interesting background stuff...

I know this from in my training material, probably it is also somewhere on the internet... Don't know directly the document...

More important is what will happen when you have a network problem between the sites... By default both sites will loose access to their data (is not the same like loose data, only the I/O is stopped)! This is because there is no majority and Lefthand cannot decide (for instance in case of a split brain situation) which site is alive...

That is why there are managers running on some of the nodes (can be started and stopped inside the CMC on the nodes). Best practice is to have an non-even amount of managers so 5 in you case.

If it is technically possible, create a third site (also on networking switch level) and install a FOM FailOver Manager. Sites A and B will have 2 managers running, the FOM will be the 5th that will decide for you which site is the survivor and maintains I/O...

If this is not possible, start 3 managers in the site which is the best protected against power, fire and other possible disasters, and only 2 managers in the other site. In case of a split brain situation, the biggest/best site will maintain I/O...

Kr,

Bart

Re: P4500 Multi Site on same subnet?

Peter J West — Tue, 19 Jul 2011 12:25:00 GMT

Thanks again Bart,

For now i've started the manager on two of the nodes at Site B although at a later date we will most likely at putting a manager in a third location.

My reason for asking the question is because nodes 4, 5 and 6 are likely to need to be shutdown at some point to be moved to another location and I wanted to basically confirm if the SAN would continue to function at Site A when the nodes are not running.

In the current setup the shutdown of the three nodes for moving will still leave us with 3 managers at Site A so that should mean that Site A (which is the primary site) should continue to function?

Thanks

Pete

Re: P4500 Multi Site on same subnet?

Bart_Heungens — Tue, 19 Jul 2011 12:31:14 GMT

Hi,

You can always see in the CMC for the Quorum number... That is the minimum amount of managers that must be running. In your case 3 managers out of 5 is a majority so that site will keep it's I/O active...

If it is possible off course you can also node per node to the new site (if networking allows...)

Kr,

Bart

Re: P4500 Multi Site on same subnet?

Peter J West — Tue, 19 Jul 2011 12:36:53 GMT

Thanks Bart - you've been a big help.

I've been really impressed by the P4000 solution - and as a nice bonus our multi-site setup obviously has a much higher capactiy than we had previously.

The next phase of our plan involves locating a backup server at the second site and using this to backup the SAN data ot a tape library. So plenty more to learn. :)

Re: P4500 Multi Site on same subnet?

Bart_Heungens — Tue, 19 Jul 2011 12:48:52 GMT

Hi,

Good to hear.

I am running P4000 already for more than 2 years at home in my test/demo environment (in the virtual version with VSA's) now with 0 seconds downtime on my volume level... It is definitely a great product!

Kr,

Bart

Re: P4500 Multi Site on same subnet?

Nott_Sys — Mon, 19 Sep 2011 16:42:15 GMT

One of the advantage of subnetting is that you can have your hosts at each site access the local nodes as P4000 Multisite Cluster is Active-Active. However, if so, then you also got to make sure that appropriate failover mechanisms are in place (ex: Virtual Manager or a Failover Manager).