topic Re: dd command question in Operating System - HP-UX

dd command question

Galen Benson — Thu, 31 Aug 2006 17:25:38 GMT

Good Day Everyone,

I'm trying to copy large files (over 2gb) from an mount point on my VA7100 to our SAN and when using the dd command, I simply get a message saying I/O Error.

We're running HPUX 11.0 & the command I'm using is dd if=/d05/exports/wdsa/060829.dmp.Z of=/newd05/exports/wdsa/060829.dmp.Z bs=1024

Our SAN vendor suggested that it may be the version of dd installed that is causing the problem, but I don't know how to find out
1. What version of dd we're running?
2. If necessary, how to upgrade this version to a version that will support files larger then 2gb.

Any help would be appreciated & as I'm leaving for the day, I can't check responses till tomorrow.

Thanks in advance,
Galen

Re: dd command question

James R. Ferguson — Thu, 31 Aug 2006 17:43:59 GMT

Hi Galen:

The first thing to verify is do you have 'largefiles' enabled on the destination filesystem. Do:

# mkfs -F vxfs -m /dev/dsk/cXtYdZ

...this output will show 'largefiles' or 'nolargeilfes' along with other attributes of the filesystem mounted on the device.

Regards!

...JRF...

Re: dd command question

A. Clay Stephenson — Thu, 31 Aug 2006 18:59:13 GMT

It's more likely that the filesystem does does support largefiles. You may need to run fsadm to enable largefiles (ie those larger than 2GiB). Although I don't think it will be necessary you can modify your dd to output to stdout and "outbushwhack" the 2GiB application limit (if it exists -- which I doubt). I'll all but sure it's the filesystem (although don't overlook ulimit and/or quotas).

if=/d05/exports/wdsa/060829.dmp.Z bs=256k > /newd05/exports/wdsa/060829.dmp.Z

Note that I changed your 1024 (bytes) to 256KiB -- that will make a huge performance difference.

Re: dd command question

Adisuria Wangsadinata_1 — Fri, 01 Sep 2006 02:01:57 GMT

Hi Galen,

You need to check it first on the mount point on the SAN whether the file system can accept the large files :

To check if large files have been activated, please enter the following command:

# fstyp -v /dev/vgxx/lvolyy|grep f_flag

If the output shows:
f_flag: 16 = largefiles
large files are activated.

If the output shows:
f_flag: 0 = no largefiles
large files are inactive.

To activate the large files, use 'fsadm' as per ACS's info. Below is the format :

# usr/sbin/fsadm -F FSType -o largefiles /dev/vgxx/lvolyy

Hope this information can help.

Cheers,
AW

Re: dd command question

Galen Benson — Fri, 01 Sep 2006 12:26:10 GMT

Thanks all,

I thought the largefiles might be the problem too but in the /etc/fstab, the entry reads as follows:

/dev/svm/dsk/Amsys3_d05 /newd05 vxfs rw,suid,largefiles,delaylog,datainlog 0 2

Shouldn't this have setup the largefiles option?

The output I get when using James' suggestion is:

mkfs -F vxfs -o iinode=unlimited,bsize=2048,version=3,inosize=256,logsize=1024,nolargefiles /dev/svm/dsk/Amsys3_d05 10485760

and when checking with the fstyp command, it shows: f_flag: 16

So it does appear to be a filesystem problem, but here's what happens when using the following command to set large files:

fsadm -F vxfs -o largefiles /dev/svm/dsk/Amsys3_d05

vxfs fsadm: /dev/svm/dsk/Amsys3_d05 is not the root inode of a vxfs file sytem

Is there something I'm doing wrong here?
Thanks again,
Galen

Re: dd command question

IT_2007 — Fri, 01 Sep 2006 12:34:51 GMT

Is it already mounted. what bdf output shows for this filesystem. don't rely on /etc/fstab and there might be typo's also.

I don't think "dsk" should exist on /dev/svm volume group. just check it out.

Once you confirm actual logical volume name and it's volume group then run fsadm command to enable largefiles option.

Re: dd command question

Fabian Briseño — Fri, 01 Sep 2006 12:35:38 GMT

Hello galen.

I dont know if this is related but I once had similar problem.

There was a file in .tar format that was being generated and every time this file was bigger than 2 GB it would fail.

The thing is, this problem was patch related, I installed a patch and my problem was solved

Re: dd command question

A. Clay Stephenson — Fri, 01 Sep 2006 12:42:21 GMT

Use the mountpoint:

Rather than:
fsadm -F vxfs -o largefiles /dev/svm/dsk/Amsys3_d05
Use:
fsadm -F vxfs -o largefiles /newd05

By the way, the largefiles/nolargefile vxfs mount options tell you absolutely nothing about the filesystem's largefile capability.

Re: dd command question

Galen Benson — Fri, 01 Sep 2006 12:55:55 GMT

Thanks ACS, that fixed it. Thanks all for your quick responses!

Re: dd command question

Galen Benson — Fri, 01 Sep 2006 12:57:58 GMT

Thanks all

Re: dd command question

James R. Ferguson — Fri, 01 Sep 2006 12:58:01 GMT

Hi (again) Galen:

Disregard the fact that '/etc/fstab' has a 'largefiles' setting for the filesystem. This only says that there it is acceptable to mount the filesystem if there are 'largefiles' in it.

The fact that the filesystem reports 'nolargefiles' signals that is does *not* support them, although the value of <16> for the 'f_flag' is consistent with a filesystem that does.

As for attempting to convert to 'largefiles", use the *raw* device file associated with the mountpoint.

Regards!

...JRF...