strange behavior, when writing to LUN on EVA6K array

[Sergey Akifiev_1]

hi.
while doing some testing we've encountered some strangeness.
in our test environment we have rx4640 server connected by its two FC cards to EVA6k disk array. each card plugged into its own fabric. this totals to 4 path for single LUN.
the box is running hp-ux 11.23.
i've looked at glance's "IO by disk" window while running `dd if=/dev/zero of=/dev/dsk/c8t0d3 bs=1024k count=$((10*1024)'. and notices strange things.
first. there were large amount of physical reads on one of the paths.
second. it seems, that sum of physical whrites equals to the amount of reads.

i'd like to know if this is normal? can it be some kind of write verification? if so, can it be switched off?

[mobidyc]

Hello,

if your filesystem is vxfs, maybe you can look the mount options:
mount -F vxfs -o convosync=direct,mincache=direct /dev/vg01/lvol /mountpoint

read mount_vxfs(1M) for more informations, always useful.

Regards,
Cedrick Gaillard

[Court Campbell]

It could be that it is reading in the pages into the buffer cache and the writing to them.

[Hein van den Heuvel]

I'm not surprised.

Did you not intend to test 'raw' write speed?
Try again targetting:

of=/dev/rdsk/c8t0d3


Hope this helps some,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting

[Sergey Akifiev_1]

i tried same test with char device node.

unusual reads gone when writing, ios spread evenly on all four paths. but i have no reasonable explanation for that. can somebody shed abit of light in this corner?

when reading ios performed on two paths only. i guess this is because EVA manages LUN on single controller only. but what about writes?

next strangenes: when load balancing policy is set to RR, _logical_ reads went on device, which wasn't mention on dd command line. but physical io went on correct path.

this all is pretty weird for me.

[Hein van den Heuvel]

Sergey,

Cedrick and Court are on to something also.

Basically you are not supposed to write to the /dev/dsk/xxx.
You use that just to create a filesystem, mount that, and then test against files on the filesystem.

In that case the mount options become relevant, and the buffering will be there, as Court already pointed out.

IMHO the system can do whatever it pleases if a non-typical (nonsense?) operation is attempted. No need to explain. no need to optimize. Maybe it will react better when writing to a file.

Somehow it decided to pre-read, merge in the data from dd, and then write. Maybe the dd buffer was unacceptable? While I like testing with 1024K block write/read for raw speed, when you go buffered it matters less and might actually be too much. Try 8kb?

Just some thoughts.
A quick google did not find an explanation, and I can not run experiements just now.

Cheers,
Hein.

[Sergey Akifiev_1]

ok. i got it.
testing read speed of "raw" disk and "raw" lvm device brings up to 140-150MB/s.