topic Re: Mirror-UX Testing in Operating System - HP-UX

Mirror-UX Testing

Scott Frye_1 — Mon, 20 Sep 2004 08:26:02 GMT

We are in the process of proving Mirror-UX to mirror between two SAN's. Currently our testing is being done with two LUN's on one SAN. I am not seeing the results I would expect, and I believe it is due to my own ignorance of how Mirror-UX works. I have LUN A mirrored to LUN B. Our test scenario has us pulling LUN B out (to simulate a downed SAN) while writing a 100 Meg file. When the file is finished, I pulled out LUN A also. Then I presented LUN B to the host. I would expect to only see a fraction of the 100 Meg file, but each time I do a 'll' on that directory, I still see a 100 Meg file.

I'm looking for a confirmation of what I should see, and what I'm doing wrong. Any additional education about Mirror-UX or pointers on where to find papers on it would be appreciated.

Thanks

Scott Frye

Re: Mirror-UX Testing

R. Sri Ram Kishore_1 — Mon, 20 Sep 2004 08:31:41 GMT

Hi Scott,

Check if these links help:
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=656293
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=190174

HTH.

Regards,
Sri Ram

Re: Mirror-UX Testing

Geoff Wild — Mon, 20 Sep 2004 09:06:02 GMT

How are you writing the file?

Your SAN may have caches the file - just cause you removed the disk from the host, doesn't mean writing has stopped - your frame will continue finishing all writes - preserving integrity of data.

100MB is a small file for this kind of test...

From "HP-UX System Administration Tasks"

"Should My Mirrored Data Be Written Simultaneously or Sequentially?

You can specify whether your mirror copies are written to disk in either a parallel (simultaneous) or a sequential fashion. A dynamic write policy is also provided.

Parallel writes are more efficient because data is transferred simultaneously to all mirror copies.

Sequential writes, on the other hand, provide for slightly higher data integrity than parallel writes. When you specify sequential writes, the data is written one copy at a time, starting the next copy only when the previous copy is completed.

Parallel writes are the default and are recommended because of the performance loss associated with sequential writes.

Dynamic writes allow the writing of mirror copies to be determined at the time of processing. If the write is synchronous, meaning that file system activity must complete before the process is allowed to continue, parallel writes are used, allowing quicker response time. If the write is asynchronous, meaning that the write does not need to complete immediately, sequential writes are selected. "

Unless you changed it, you are writing parallel...

IMHO - you have tested Mirror UX - and it is working as expected.

Rgds...Geoff

Re: Mirror-UX Testing

Sundar_7 — Mon, 20 Sep 2004 10:51:29 GMT

If I understand your question, you start writing to the filesystem while you pull out LUN B. Now only LUN A is available. When the file write is complete, you are pulling out LUN A as well and then inserting LUN B.

Hmm...if you ask me, this doesnt sound like an "ideal" test :-). One reason being that during the period between pulling out LUN A and plugging the LUN B, the filesystem is on a disk that is not available !. I would guess there will be some complications due to this. This could possibly explain the output of "ll" command.

Try this. Keep LUNA and B sync. Pull out LUN B. Create a file on the filesystem. Pull out LUN A. Plug back LUN B. Now do a "ll" and see if you find the file you created when LUN B was not available to the system.

If you still find the file, it is probably due to some kind of INODE caching. I would not like to think system can switch between the meta data information on two different disks dynamically :-)

Re: Mirror-UX Testing

Steven E. Protter — Mon, 20 Sep 2004 10:59:21 GMT

The only valid test is to complete the setup and then pull the cables connecting to one of the SAN's.

To properly mirror the two logical volumes the command is:

lvextend -m 1 /dev/vg00/lvol8 /dev/dsk/c1t1d0'

use real names.

If that process did not complete then you may need to do the mirror again.

SEP

Re: Mirror-UX Testing

Mark Greene_1 — Mon, 20 Sep 2004 11:22:58 GMT

What model SAN are you using, and what RAID level are the LUNS on the SAN?

mark

Re: Mirror-UX Testing

Scott Frye_1 — Mon, 20 Sep 2004 11:23:09 GMT

We are using an EMC SAN (CX600).

We tried the scenario someone else pointed out. We only had LUN A assiged to the host. We wrote the 100 Meg file (10 minutes to write), did an 'll' to verify it was there then removed LUN A. We then presented LUN B to the host, did an 'll' and still saw the file.

Again, I would have expected to see nothing in LUN B.

Re: Mirror-UX Testing

Scott Frye_1 — Mon, 20 Sep 2004 11:23:11 GMT

We are using an EMC SAN (CX600) and HP-UX 11i

We tried the scenario someone else pointed out. We only had LUN A assiged to the host. We wrote the 100 Meg file (10 minutes to write), did an 'll' to verify it was there then removed LUN A. We then presented LUN B to the host, did an 'll' and still saw the file.

Again, I would have expected to see nothing in LUN B.

Re: Mirror-UX Testing

Scott Frye_1 — Mon, 20 Sep 2004 11:28:03 GMT

Raid 5

Re: Mirror-UX Testing

Kent Ostby — Mon, 20 Sep 2004 11:34:43 GMT

Scott --

I'm not sure if this is a factor or not, but keep in mind that HP-UX keeps a lot of the information about files that have been recently used in memory.

You're not REALLY looking at LUN B.

You're looking at the filesystem that you wrote to. It may be looking at LUN B or it may be referring to what it knows to be going on in memory.

Try copying the file now that you only have LUN B in the SAN and see what happens. It still might work since if the file is in memory, the system can copy it out of memory.

Best regards,

Kent M. Ostby

Re: Mirror-UX Testing

Mark Greene_1 — Mon, 20 Sep 2004 11:43:18 GMT

Scott,

We have the same hardware, so I'm familiar with the systems.

When you say "We then presented LUN B to the host", this isn't a single step process. Here's what I would do to make that happen:

- on the HP run ioscan -fnC disk and route the output to a printer or file for later comparison

- unmount the file systems

- run vgexport for the volume group in question.

- in navisphere, remove LUN A and add LUN B in the host storage group

- on the HP run powermt display dev=all and route the output to a file or printer for later comparison

- run ioscan -f

- run insf -e -C disk

- run ioscan -fnC disk and compare this output from the previous run. You should see the SCSI address for the virtual disk change. If not, you have other problems, either with the SAN or powerpath.

(hmmm... I'm assuming you are running powerpath and not just using alternate links, yes?)

- run powermt check. there will be no output, this is normal

- run powermt config. there will be no output, this is normal

- run powermt display dev=all and compare to the previous run. The SCSI paths should be the only change.

- vgimport the volume group

- mount the file systems

HTH
mark

Re: Mirror-UX Testing

Sundar_7 — Mon, 20 Sep 2004 12:37:05 GMT

Mark - I dont believe those steps are required. If the disk is already visible to the host and if you are just pulling it out and plugging it back again, kernel automatically performs the scan of the devices and update the status as CLAIMED. In the worst case, you might require a ioscan -fnC disk to refresh the status.

But as I said earlier in my post, I believe the problem is with the procedure followed to test Mirror-UX and not with MirrorDX/UX.

Scott - Try creating a file in LUN A when LUN B is pulled out. Pull out LUN A now and make LUN B visible to the host. Now do a "ll", I am positive you will see the file you just created when only LUN A was online.

A more realistic test would be to just pull out one of the drives and ensure you are still able to access the filesystem.

Re: Mirror-UX Testing

Mark Greene_1 — Mon, 20 Sep 2004 12:44:36 GMT

Sundar,

He is using a SAN, not direct attached disk. The physical drives are carved into virtual Logical drives. The only physical connections are the SCSI paths. Added to this is the fact that he's testing using LUNs on the same SAN. The kicker is that his test process is going to be different when he moves to the environment he wants: two LUNs, each on a different SAN.

If he's using powerpath (software from EMC), then he has to accound for powerpath masking the SCSI paths as well.

mark

Re: Mirror-UX Testing

Scott Frye_1 — Mon, 20 Sep 2004 12:49:55 GMT

Sundar and Mark are beginning to feel my pain. Mark, we are not using PowerPath. We are using PVLinks.

To your point Mark, when I present and take away the LUN from the host (storage group), I see the change in ioscan without having to do anything else. We checked that out to make sure we were writing to only one LUN.