Speed difference between dsk/rdsk - why?

[Laszlo Csizmadia]

There is a significant read difference between reading from rlvol or lvol (rdsk/dsk) device:
$ timex dd if=/dev/dsk/c4t7d2 of=/dev/null bs=64k count=1000
1000+0 records in
1000+0 records out
real 4.11
user 0.00
sys 0.90

$ timex dd if=/dev/rdsk/c4t7d2 of=/dev/null bs=64k count=1000
1000+0 records in
1000+0 records out
real 0.63
user 0.00
sys 0.05

Could somebody explain me where is the difference comes from?
BTW why is it possible to dd from dsk/lvol? I thought that only caracter devices can used for this. Is this an HPUX feature or I simple don't understand which device is used for what?
Thanks in advance.

[Massimo Bianchi · Accepted solution]

Hi,
you can dd either from disks or from lvol.

Why character device are faster?

quoting from "man dd"

If if or of refers to a raw disk, bs should always be a multiple of
the sector size of the disk. By default, bs is 512 bytes. If the
sector size of the disk is different from 512 bytes, bs should be
specified using a multiple of sector size. The character special
(raw) device file should always be used for devices.



I think that the reason could lay in the bypassing of the buffer cache for raw device.

Massimo

[Robert-Jan Goossens]

Hi,

Using the raw device is faster, you will miss the lvm overhead.

Hope it helps,

Robert-Jan.

[Steve Lewis]

mmm... c4t7d2 looks like a disk array LUN. The second operation probably had the data in the array cache so it wouldn't matter which device you used for the second test.

you can dd from any file - its UNIX after all.

[Massimo Bianchi]

To Steve:
reading from a raw disks bypasses the buffer cache, so data was not already there.

I also know that reading from raw is faster, and my above guess is the only exaplanation i can think of.

Massimo

[Laszlo Csizmadia]

Steve,
right this a storage disk but it doesn't matter. For local disk the difference is the same.

Massimo,
you said that in case of raw disk buffer cache is used. Why is it not used in case of block devices(dsk)?
As you quoted from man dd:
"The character special(raw) device file should always be used for devices."
It means that block devices couldn't even used for dd, doesn't it?
BTW 64k is a multiple of 512byte, so dd above was ok in that respect.

I hardly believe that lvm overhead can cause this.
What about readahed cache? Does it matters?

[Massimo Bianchi]

Maybe i explained bad.

When you use raw disk buffer cache is BYPASSED, is NOT USED. you do not have to transfer data to memory and after to destination.

When you use the block special file, all data goes through many other layer, thus slowing down.

So, when using dd, for a faster transfer raw disk, aka character device, are suggested.

Massimo

[Massimo Bianchi]

"man disk"


Block-special access
Block-special device files access disks via the system's block buffer
cache mechanism. Buffering is done in such a way that concurrent
access through multiple opens and mounting the same physical device is
correctly handled to avoid operation sequencing errors. The block
buffer cache permits the system to do physical I/O operations when
convenient. This means that physical write operations may occur
substantially later in time than their corresponding logical write
requests. This also means that physical read operations may occur
substantially earlier in time than their corresponding logical read
requests.

Block-special files can be read and written without regard to physical
disk records. Block-special file read() and write() calls requiring
disk access result in one or more BLKDEV_IOSIZE byte (typically 2048
byte) transfers between the disk and the block buffer cache.
Applications using the block-special device should ensure that they do
not read or write past the end of last BLKDEV_IOSIZE sized block in
the device file. Because the interface is buffered, accesses past
this point behave unpredictably.

Character-special access
Character-special device files access disks without buffering and
support the direct transmission of data between the disk and the
user's read or write buffer. Disk access through the character
special file interface causes all physical I/O operations to be
completed before control returns from the call. A single read or
write operation up to MAXPHYS bytes (typically 64 Kbytes or 256
Kbytes) results in exactly one disk operation. Requests larger than
this are broken up automatically by the operating system. Since large
I/O operations via character-special files avoid block buffer cache
handling and result in fewer disk operations, they are typically more
efficient than similar block-special file operations.

[Steve Lewis]

You all completely mis-understood what i was talking about. By cache I meant the disk array controller cache, not the buffer cache.

Surely everyone knows that io to and from /dev/dsk devices gets buffered through the o/s and filesystem i/o goes through the buffer cache.

[Steve Lewis]

well done Massimo for clarifying the issue from the man page - i didnt see your last posting.