topic Re: VxFS defragmentation in Operating System - HP-UX

VxFS defragmentation

Kevin Bingham — Thu, 09 Sep 2004 10:34:10 GMT

Hi everyone,

I am running 11.11 on a L1000 machine which has 2 disks, 1 for operating system and supporting software, the other for application development etc.

The OS disk hardly changes, but the AppDev disk has 100's of updates/creates/deletes per week. I have not bothered with trying to tune this disk's performance as yet, and have to admit to being new to this area of HP-UX Sys Adm.

So, I was hoping that the kind people who frequent this forum would provide me with an explicit recipe to defragmenting (if necessary) this disk, and give any other tuning tips as you deem fit. I hope I have supplied enough detail, if not let me know and I shal try to answer any questions as necessary.

The research I have done so far seems to indicate that I need to use Veritas software (fsadm_vxfs, etc), and this is what I find under swlist
"Base VERITAS Volume Manager 3.2 for HP-UX"

Is this the software being referred to?

Thanks in advance for your detailed responses... assume I know very little on this subject.

Regards
Kevin

Re: VxFS defragmentation

Pete Randall — Thu, 09 Sep 2004 10:41:41 GMT

Kevin,

I routinely (via cron) defrag all my file systems using the following little script:

for i in `bdf -l |grep -v Filesystem |awk '{ print $6 }'`
do
if [ $i != "/stand" ]; then
echo "Re-Org'ing " $i
fsadm -F vxfs -d -D -e -E $i
fi
done

I can't say that it makes a huge difference but it certainly doesn't seem to hurt.

Pete

Re: VxFS defragmentation

RAC_1 — Thu, 09 Sep 2004 10:42:49 GMT

With just one disk for application, you have a very little to do. What application is this?? Depending on the application and the data it accesses, it is going to be difficult to know which data is accessed most and which least.

The defragmentation can be done with fsadm -d ad -e options. read the man pages of vxfs_fsadm. If you do not have OnlineJFS(swlist -l product OnlineJFS), you will have to unmount the FS and do the defragmentation work. But to work it, you should have some free space on the FS.

A better defragmentation would be doing a full cold backup and restore.

Stop all applications accessing the FS.
Back it up.
newfs on the FS and restore.
Start the application.

Also you may want to look at your buffer cache setting.

Anil

Re: VxFS defragmentation

Sundar_7 — Thu, 09 Sep 2004 10:43:28 GMT

fsadm_vxfs that comes with Base VERITAS volume Manager cannot do defragmentation.

You need to purchase and install a product called "Online JFS".

Online JFS allows you to
1) Dynamically reduce/extend the filesystem
2) defragment the filesystems online
3) Create a snapshot of the filesystem

Which OE did you install ? - If you have installed Mission Critical/Enterprise OE, you might already have Online JFS.

# swlist -l file OnlineJFS

IF yes then you can

# fsadm -F vxfs -D -E -d -e /

The above command will defragment the filesystem for you.

-- Sundar.

Re: VxFS defragmentation

Kevin Bingham — Thu, 09 Sep 2004 10:45:43 GMT

Thanks, that looks very promising... When I run fsadm, on one of the filesystems, i get the following output:

{root} /> fsadm -F vxfs -d -D -e -E /apps2
fsadm: You don't have a license to run this program
{root} />

I am guessing this means installing something... any tips before I start digging?

Re: VxFS defragmentation

A. Clay Stephenson — Thu, 09 Sep 2004 10:49:10 GMT

This has nothing to do with VxVM but only pertains to fsadm -F vxfs. THe man pages for fsadm_vxfs should be all to need. Just invoke fsadm -F vxfs -d -e /fsmountpoint and you are good to go. This can be done while the filesystem is mounted.

In general, don't expect big gains from running this. Vxfs handles fragmentation well and in the UNIX world many, many file changes per second is the norm. Any filesystem that could handle that would simply never make it to prime time. There are a few corner cases where defragging noticeably helps but most of the times any gains are difficult to measure and extremely difficult to perceive.

Re: VxFS defragmentation

Pete Randall — Thu, 09 Sep 2004 10:50:19 GMT

Kevin,

Yes - I missed that you only had the Base version. In order to run de-frag at all, you need to purchase and install the Online version:

http://www.software.hp.com/portal/swdepot/displayProductInfo.do?productNumber=B3929CA

Pete

Re: VxFS defragmentation

Kevin Bingham — Thu, 09 Sep 2004 10:53:21 GMT

{root} /> swlist -l product OnlineJFS
# Initializing...
# Contacting target "hp2"...
ERROR: Software "OnlineJFS" was not found on host "hp2:/".
{root} />

Is there any other way to do the defrag without using fsadm/Veritas.

Perhaps a little more background might help. We use the L1000 machine for porting work only. The software we develop is generated from a Model-Based development tool, and we generate the code for Win/Oracle, Win/MS-Sql, Unix/Oracle and MVS/DB2, all of which are automatically compiled and linked as part of our development process. So we seldon log on to the L1000, except for testing our software, most of which happens on the Win/Oracle platform... Recently, we have noticed that the compiles and links of our Unix/Oracle code is taking longer than it used to, and I figured that it is probably down to the HD needing some TLC, hence my posting...

We have a Dev Partner relationship with HP since our software is certified to run on HP h/w, and as such, could possibly get Veritas, but at present it does not seem that we have it.

Anyone who is interested (curious) in our software can look at our website (www.iet.co.uk)

Re: VxFS defragmentation

Kevin Bingham — Thu, 09 Sep 2004 10:55:57 GMT

Also, since this machine is only used for porting work, it has no Tape Drive and does not get backed up, so backing up and then doing a "newfs" isn't really an option I want to follow right now... thanks anyway.

Re: VxFS defragmentation

Sundar_7 — Thu, 09 Sep 2004 10:59:20 GMT

I dont believe there is any other way out to defragment the filesystem (not that I am aware of :-) , except installing OnlineJFS.

Re: VxFS defragmentation

Sanjay_6 — Thu, 09 Sep 2004 11:02:31 GMT

Hi Kevin,

Doesn't look like you have OnlineJFS license on the box to do the defrag of the filesystem. You need to have onlineJFS on the system. OnlineJFS is a $$ product and is sold seperately.

Hope this helps.

regds

Re: VxFS defragmentation

Kevin Bingham — Thu, 09 Sep 2004 11:08:47 GMT

I may have found OnlineJFS on the bundle of CD's that came with the machine... stand by...

Re: VxFS defragmentation

Pete Randall — Thu, 09 Sep 2004 11:12:37 GMT

Kevin,

I can pretty much guarantee that Online JFS is on the CDs, but you'll need a codeword to unlock and install it.

Pete

Re: VxFS defragmentation

Kevin Bingham — Thu, 09 Sep 2004 11:19:38 GMT

Well, I have managed to install OnlineJFS, so now I an running the fsadm command that everyone suggested I try:

Here is the output so far, does this look badly fragmented, how long should it take to run?

{root} /homeroot> swlist -l product OnlineJFS
# Initializing...
# Contacting target "hp2"...
#
# Target: hp2:/
#

OnlineJFS B.11.11 Online features of the VxFS File System
{root} /homeroot> fsadm -F vxfs -d -D -e -E /apps2

Directory Fragmentation Report
Dirs Total Immed Immeds Dirs to Blocks to
Searched Blocks Dirs to Add Reduce Reduce
total 3863 2281 2895 28 46 66

Re: VxFS defragmentation

Sundar_7 — Thu, 09 Sep 2004 11:27:04 GMT

How much time it is supposed to take ? - the best possible answer is "it depends"
:-).

It depends on the size of the filesystem, how fragmented it is, the amount of activity in the system etc.

Re: VxFS defragmentation

Geoff Wild — Thu, 09 Sep 2004 11:28:17 GMT

Doesn't look to bad...

I run the following weekly on my servers:

cat /usr/local/bin/fsadm.defrag
#!/bin/sh
# defrag all mounted file systems
LOG=/tmp/fsadm.defrag.log
if [ -f $LOG ]
then
mv $LOG $LOG.old
fi
cat /dev/null > $LOG

for i in `mount -l | grep -v stand |awk '{print $1}'`
do
echo "defraging " $i >> $LOG
fsadm -F vxfs -d -D -e -E $i >> $LOG
done

Rgds...Geoff

Re: VxFS defragmentation

Kevin Bingham — Thu, 09 Sep 2004 11:29:22 GMT

bdf output shows:
/dev/vg01/lvol1 9072640 7593111 1387626 85% /apps2
and this is the only *active* process on the machine, no users logged on...

Re: VxFS defragmentation

Pete Randall — Thu, 09 Sep 2004 11:30:15 GMT

Kevin,

It's probably too early to tell. A look at the man page tells us "If the totals in the columns labeled "Dirs to Reduce" are substantial, a directory reorganization can improve performance of pathname resolution."
The examples show "Dirs to Reduce" in the 2500 to 3000 range, far above your 46. This may run for a long time but I don't think you're going to see any substantial improvement.

You might want to start looking at other performance metrics of this machine to see what is slowing it down.

Pete

Re: VxFS defragmentation

Kevin Bingham — Thu, 09 Sep 2004 11:32:14 GMT

Ok, now that I have OnlineJFS and fsadm is running where else can I start looking for some perfomance improvements? Perhaps I should start a new thread for this one?

Re: VxFS defragmentation

Pete Randall — Thu, 09 Sep 2004 11:37:26 GMT

Probably a new thread would be best.

Pete