topic Re: Multiple Servers access SAN LUN in Operating System - Linux

Multiple Servers access SAN LUN

barakanooz — Fri, 27 Jul 2007 11:22:22 GMT

I have a HP MSA1000 storage unit and multiple hp servers setup on a fiber network.
I HAD one of the servers, hp#1 directly mounted to the various LUNs setup on the MSA1000 unit
Then HAD the other hp machines access those logical drives on the MSA1000 by mapping to NFS shares from hp#1

Problem: backups where taking FOREVER going over the NFS mounts so..

I changed the setup so that each hp machine now mounts directly to the LUN it needs on the MSA1000 unit, while maintaining the direct mounts for hp#1 which puts everything on tape.
Works great for backups! MUCH Faster!

BUT.. now if I do a df -h on each of the hp machines the LUN it is mounted to on the MSA unit shows up with the wrong information.
For example hp#1 and hp#2 both directly connect to a LUN on the MSA1000 unit. When I run a df -h on each of these machines the results are:
.............Filesystem Size Used Avail Use% Mounted on
hp#2 - /dev/sdb1 429G 324G 84G 80% /n03
hp#1 - /dev/sdc1 429G 48G 359G 12% /r03

Also, I tried a "du . -sh" to see the size reported and this is what I get (this result is the correct space used):

hp#1 - 325G .
hp#2 - 325G .

The size of the LUN is correct but the other stats are skewed. And now my backups from hp#2 to the LUN fail because the system doesn't think there is enough drive space.

I don't want to switch back to using NFS because it is so slow and I don't understand why I would have to. The file system on the LUN on the MSA is ext3.

THX

Re: Multiple Servers access SAN LUN

Ivan Ferreira — Fri, 27 Jul 2007 15:44:37 GMT

If I understand correctly the problem, by now, you should already have all corrupted data.

You cannot mount the same LUN in more than one server for Read/Write access without a clustered file system, like GFS or OGFS.

The cluster file system coordinates the access between nodes, without it, the hosts cannot know if the data is accessed by another host, and both will be updating metadata without coordination, leading in a corrupted file system and invalid data.

Re: Multiple Servers access SAN LUN

skt_skt — Fri, 27 Jul 2007 20:23:46 GMT

"changed the setup so that each hp machine now mounts directly to the LUN it needs on the MSA1000 unit", .. i did not catch it proper.. I would say u might have lost all the data(corrution) by now.

Look for old backup which u had before doing this change. start working to rivert the changs.Any way take FS && RAW backup from both the nodes.Recall all the old tapes (if required to restore).

I had a similar issue in HP-UX last week; by mistake an FS left unmounted on another node.We lost everything.

Re: Multiple Servers access SAN LUN

Patrick Terlisten — Sat, 28 Jul 2007 03:36:54 GMT

Hello barakanooz,

if you want to access a filesystem with more than one node, you need a filesystem which supports parallel access, like GFS or the free OCFS2. Other filesystems like ext2/3, reiserFS or XFS don't have a proper locking mechanism, so there is a high risk of data corruption.

Best regards,
Patrick

Re: Multiple Servers access SAN LUN

Yogeeraj_1 — Sat, 28 Jul 2007 04:19:35 GMT

hi,

You should also not mix backup traffic with your data traffic!

Normally, dedicated HBA card are used for backup.

You can avoid these types of problem by doing an appropriate Zoning at the SAN switch level.

hope this helps too!

kind regards
yogeeraj