Operating System - HP-UX
1836627 Members
1918 Online
110102 Solutions
New Discussion

Need to move data from one filesystem to another

 
SOLVED
Go to solution
John King_12
Frequent Advisor

Need to move data from one filesystem to another

Hi,

I am fairly new to the HP-UX O.S so please bare with me. I am managing an rp7410 configured inot two seperate partitions, running HP-UX B11.11 with latest (recommended patches)
I have been asked by my client to move data (application) from one filesystem to another new filesystem. And rename the new filesystem with the old filesystem name.
The application resides on vg00,lvol09. The new filesystem will be vg02 and I would like to use lvol09 again for the logical volume name.
I was considering a couple of options available to me.
The first, use vxdump and dump the application filesystem to tape, then umount the application filesystem, then deactive lvol09 on vg00 via lvchange, create the new vg and lvol09 (again) and remount.

The second option would be to Create the new vg and lvol, then dd old application filesystem to new vg/lvol, then via lvchange, deactivate the old application filesystem, umount the new app filesystem and remount.

I know I have left out some crucial steps here.
If at all possible and if you are not to busy, could someone please tell me the exact steps I will need to follow. Or purhaps point me to a previous posting or procedure doc.

Points will be assigned upont successful move of data.
Thanks in advance and have a great day!

John K
19 REPLIES 19
Geoff Wild
Honored Contributor
Solution

Re: Need to move data from one filesystem to another

You can use vxdump/vxrestore - and you don't need to use tape:

vxdump -0 -f - -s 1000000 -b 16 /olddata | (cd /tempmountfornewdata ; vxrestore rf -)

As far as vg's go - doesn't matter - if room in vg02, then place it there - or create a new vg if you like.

Rgds...Geoff
Proverbs 3:5,6 Trust in the Lord with all your heart and lean not on your own understanding; in all your ways acknowledge him, and he will make all your paths straight.
MarkSyder
Honored Contributor

Re: Need to move data from one filesystem to another

Hi John.

First of all, I'm curious about moving from vg00 to vg02: vg00 is usually operating system stuff unless you've only got one disc. What is the name of the filesystem?

Secondly, the best way to move will probably depend on the size of the filesystem. What size is it?

Mark Syder (like the drink but spelt different)
The triumph of evil requires only that good men do nothing
Mark Greene_1
Honored Contributor

Re: Need to move data from one filesystem to another

I'd stay away from dd, too much room for error with the LVM info there.

The tape option is the least risky, but also probably the slowest. The other obvious low-risk option it to create the new vg and lv, create the new file system, and then cp the files from one filesystem to the other. This should be a bit faster than tape, too.

mark
the future will be a lot like now, only later
Michael_356
Frequent Advisor

Re: Need to move data from one filesystem to another

Hi there,

how about:

cd /
mkdir olddir
mkdir newdir

mount /dev/vg00/lvol09 /olddir
mount /dev/vg00/lvol02 /newdir
# (maybe you have to unmount these lvols before do above)

cp -Rp olddir/* newdir
umount /olddir
umount /newdir

mount /dev/vg00/lvol02 /name_of_application

mount /dev/vg00/lvol9 /whereever_you like

regards

Michael
TwoProc
Honored Contributor

Re: Need to move data from one filesystem to another

John,

Micheal's response (above) gets my vote for simplicity since you're not too familiar with things. His method above is the easiest method
I can think of - especially since you'll be performing the move as root - and he's given you commands that will preserve both ownership and permissions.
We are the people our parents warned us about --Jimmy Buffett
John Waller
Esteemed Contributor

Re: Need to move data from one filesystem to another

Hi John,
Couple of points. Is this data on vg00/lvol9 raw data or is it standard files/directories. If files/directories why complicate the matter with dd or vxdump, use a standard cp or cpio. The way I would attack this is as follows:

1 lvcreate -L vg02 (This will create a lvol in vg02 with the next sequence number)
2 newfs -F vxfs /dev/vg02/rlvol where is the lvol number created above.
3 mkdir /stage
4 mount /dev/vg02/lvol /stage
5 cd / where / is mount point of /dev/vg00/lvol9
6 find . | cpio -pdvmua /stage
7 umount /stage
8 umount /app
9 mount /dev/vg02/lvol /app
10 modify /etc/fstab to /dev/vg02/lvol is the correct lvol for the /app filesystem.

You will need to make sure that you application is not running whilsy you perform your cpio as you will possibly get data corruption.
John King_12
Frequent Advisor

Re: Need to move data from one filesystem to another

Hi Mark,

I realize that vg00 is typically used for "root filesystem(s) however we are somewhat limited for space.

The new vg02 will be on an Hitachi SAN.
Then name of the filesystem will be apps and mounted to /apps.

The amount of data I will be moving is 6.655GB. Current filesystem is 10GB total in size and the new vg02 will be of the same size.

I am at this point leaning towards Michael's post or Geoff's post.

Thanks!



Geoff Wild
Honored Contributor

Re: Need to move data from one filesystem to another

Just from experience, I have done san migrations in the past, and vxdump is by far the quickest.

Rgds...Geoff
Proverbs 3:5,6 Trust in the Lord with all your heart and lean not on your own understanding; in all your ways acknowledge him, and he will make all your paths straight.
John King_12
Frequent Advisor

Re: Need to move data from one filesystem to another

Hi John Joubert,

I have been monitoring this Admin Forum for almost a year now, and have been noticing the different responses with regards to data movement, "which is faster?". Do you have any comments regarding Geoff's last post?

Initially I was considering using cp -Rp but have heard of issues of speed when writing to SAN devices.

Regards,

John K
John Waller
Esteemed Contributor

Re: Need to move data from one filesystem to another

I used to use cp -rp but after reading a post myself I started using the cpio option which I have found is faster than cp, but I don't have a SAN so perhaps another command may be faster for SAN's
John King_12
Frequent Advisor

Re: Need to move data from one filesystem to another

Hi John Waller,

The filesystem was configured as a JFS filesystem, with large files support. The reason why I am asking which movement method I should use is strictly due to performance reasnons. I have some backups I will be performing on the system in question as well as additional hardware to add so from a time stand point, I would like the move to be quick as possable.

Thanks!

Regards,

John K

John King_12
Frequent Advisor

Re: Need to move data from one filesystem to another

Hi,

Just to let you know, I will assign full points upon successfull movement of data. This will be occuring within the next week.
If anyone has anything else which may have been overlooked, please feel free to add.
My thanks to everyone who responded I have awarded some points for verifying or commenting on other forum members suggestions. Thanks for your replies!
This is a great forum. Keep up the fantastic
work.
Again, I will close this post and award full points when work has been completed
Chris Vail
Honored Contributor

Re: Need to move data from one filesystem to another

One of my favorite ways to move data like this is with tar. From the source directory:
tar cvf - .|(cd $DESTDIR; tar xvf -)

This causes tar to read the current directory and standard output, but also change do the destination directory then read from from standard input.

Tar and cpio are both pretty quick. I've always found tar quicker but also more cpu intensive.


Chris
MarkSyder
Honored Contributor

Re: Need to move data from one filesystem to another

I agree with Chris - I've used tar for this sort of work. But I'm intrigued that Chris describes tar as being quick: HP-UX System Administration by Marty Poniatowski describes tar as being "as slow as molasses"!

Speed considerations aside, I think tar is a user-friendly way of moving the data.

Mark
The triumph of evil requires only that good men do nothing
TwoProc
Honored Contributor

Re: Need to move data from one filesystem to another

John,

I admit that like someone else's post - I usually use cpio - because, I admit - I like the output on the screen better :-) . But, I think that cp is going to provide good throughput.

Alas, I'm not familiar with vxdump, and haven't given it much thought - however with Geoff's mention that he's done a number of SAN migrations with it - I'll definitely be looking it over and evaluating it for my next SAN migration. So, my answer is - it's easy enough to test these tools, and get some rough numbers - so I think you should give vxdump a try. My earlier recommendation was based on the fact that it sounded like you're a bit new to this, and it would be the easiest in concept to use and gauge how you're doing during the process.

Best is to do some sample tests, and measure your throuput - you can do this even on "open" or hot database file. Of course, the resultant files won't be any good - it's a throwaway, but you should be able to go into a subdir and get some time estimates which will tell you which way is faster.

That being said - there's a lot consider when you created your new volume on your Hitachi SAN. Did you expose all of your LUNS to your HP box as one big LUSE ? I find that the Hitachi techies really like this method - but fail to consider that HPUX is going to give you much better throughput to multiple LUNS interleaved via distributed stripes (to the PE size) to your SAN device. I find that Hitachi's implementation method may vary from what HP might do implementing an Hitachi rebranded device (XP1024, XP512). I generally use OPEN-9 or OPEN-E and combine at most two together in a LUSE configuration for high I/O database areas. Your access to lot's of devs and interleaving them is going to provide you a fair amount of I/O bandwidth to your storage frame, but could be a limitation without. Hopefully you installed a healthy amount of I/O cards to make it happen.

Anyways... cpio would be...
cd /source_mnt_point
find . | cpio -pdmvu /targe_mnt_point

another interesting that may work well (you'll need to time it) that I use most often between servers (not on the same server - which is what you're doing) is tar. For you, it would look like...

cd /source_mnt_point
tar cvf - . | (cd /target_mnt_point ; tar xvf - )

to get timings - use the "time" command in front of any command you want to run.

Just be careful during your tests not to trash your production databases - right?
Since you know you're kind of new at this, go and find an old test box to play with. I've kept old boxes just for this very purpose for our newest sys admins and they admit that without them they would have never learned nearly as much. You need a place to be able to make mistakes and fix them without breaking production.

Lastly, keep in mind that if you've got one of the Hitachi systems, they can easily handle a great deal of concurrency. This means that you can copy with many jobs at once. I'd probably restrict this to one job per file system - because you don't want to get your database files all fragmented up ( I do realize you're doing just one filesystem ). But, just a general question - is the best solution just one file system? Is this just an app - or an app with databases or binary files in it that are database-like?
We are the people our parents warned us about --Jimmy Buffett
John King_12
Frequent Advisor

Re: Need to move data from one filesystem to another

Hi John Joubert,

Unfortunately this is a productions system and as a result, I am limited to the testing I can perform. Also continuing on the downside, we do not have a HP-UX system currently connected to any SANs. This will be a first. I have a fair bit of experience with the Tru64 O.S, so if this was the same O.S, I would not have bothered posting at all.

Conserning the SAN disk layout. We will be getting a total of 80GB for the first partition of the rp7410. This 80GB will be divided into 2 LUNS (RAID5). I have configured SAN storage for a few years (hardware level) and I have always understood the principle of "the more drives provided in a RAID set (RAID5) the better the performance" since you are allowing more drives to participate in the read/write request.

Also since the new storage will be RAIDed, I will not use any O.S disk striping. You are defeating the whole idea of a SAN, RAID technology, and the performace benefits doing this. I have also read from previous posts, other forum members sharing this same opinion.

I will also choose to disable bad block allocation when creating the lvol as per previous postings on this forum.

Currently there will be one FCHBA per partition. The project I am working with have a limited budget so the full redundancy was not an option. Also if I am not mistaken, we would also need a second piece of software with license to make use of the full redundancy option. i.e. Secure path, Power path etc. So speaking from a performance setup, this is not an ideal configuration, however I will make it perform as best I can.

Thanks for your testing suggestions, I will try to implement some of them.

Regards,

John K.
Chris Vail
Honored Contributor

Re: Need to move data from one filesystem to another

Mark: When writing to/from disk devices themselves, cpio, vxdump, and dd are all faster. But in reading/writing to/from standard i/o, then tar is more efficient. Using standard i/o means that the CPU and memory are being used as a big buffer. This allows more intelligence to be used by the subsystems, which increases spead. Its been a while since I benchmarked it (and I benchmarked it on SCO, so take this FWIW), so can learn something new. But tar SCREAMS when using standard i/o. Its even pretty good going across an NFS mount point.
One of tar's nicer attributes is that it is nearly universal: a tar dump made on one machine (like HP) can usually be read on another (like IBM or Sun). This isn't true with most other forms of backup.

Chris
John King_12
Frequent Advisor

Re: Need to move data from one filesystem to another

Hi,

All work has been completed successfully. Michaels response was the solution of choice. I have awarded full points to all solutions which would have worked.
Thanks to all who responded to this post. Keep up the great work! This post will be closed.
John King_12
Frequent Advisor

Re: Need to move data from one filesystem to another

Michaels solution was chosen. Thanks again


Regards,

John K