topic Re: High Interrupt State in Operating System - OpenVMS

High Interrupt State

rison so_1 — Wed, 21 Mar 2007 21:25:04 GMT

System: ES45 and DS20E in cluster
OS: OpenVms V7.3-2
Storage: HSG80

Problem: Very high Interrupt State when deleting files on HSG80.

Would like to know why and what is causing it. And maybe the fix.

Thanks

Re: High Interrupt State

Hein van den Heuvel — Wed, 21 Mar 2007 21:56:09 GMT

Is this an MSCP served disk, or direct attach, or perhaps intended to be direct but actually being served?

Any more details on that delete?

Does a single file trigger this?

- I'm thinking of directory reshuffling when large amount of files are deleted from large directories (> 1000 blocks)

Does any size file seem to trigger this ?

- I'm thinking of DELETE/ERASE activity

While Interruts stack time is of course naturally associated with Interupt, the biggest contributions often come from LOCK activity, notably in a cluster. Do you see a reason why you would have high lockig when those deletes happen?

>> Would like to know why and what is causing it. And maybe the fix.

Is this a serious, repeated, problem for your system or a curiousity? I have a hard time imagining an applications deleting so much that it becomes a problem. Eventually you would run out of files no? :-).

If it is repeatable, can you correlated some activities? T4 would be the preferred tool for that.

http://h71000.www7.hp.com/OpenVMS/products/t4/index.html?jumpid=reg_R1002_USEN

fwiw,
Hein.

Re: High Interrupt State

John Gillings — Wed, 21 Mar 2007 21:58:26 GMT

Hi Rison,

Can you be more specific? How are you deleting files, and how many (command and output)?

Are you using DELETE/ERASE or is the volume set to erase on delete? Check SHOW DEVICE/FULL. If it's enabled and you don't want it enabled use:

$ SET VOLUME/NOERASE_ON_DELETE

to turn it off.

How high is "very high", and for how long?

For an authoritative answer, please log a case with your local customer support centre.

Re: High Interrupt State

rison so_1 — Wed, 21 Mar 2007 22:14:36 GMT

It is a MSCP Served disk.

I'm trying to delete all files in a directory using just DELETE, and Yes there is a sud-directory has size > 2000 block.

The high interrupt state can be reproduced by deleting another similar directory with size > 2000 block.

When delete is happening, the system runs very slow, interrupt state goes up to 180 to 200 on a 3 CPU server.

Is my system running out some resources ?

Re: High Interrupt State

John Gillings — Wed, 21 Mar 2007 22:41:34 GMT

Rison,

>I'm trying to delete all files in a
>directory using just DELETE, and Yes there
>is a sud-directory has size > 2000 block.

This is a design flaw in OpenVMS. DELETE scans files in the absolute worst possible order. For large directories, performance is terrible.

To delete an entire directory, use DFU (on the freeware) - the command is DELETE/DIRECTORY. It will be very fast with minimal interrupt activity. If you want to delete a large number of files, but not all, generate a list of the target files with DIRECTORY/COLUMN=1 or DIRECTORY/NOHEAD/NOTRAIL, sort it into inverse alphabetical order and tack a DELETE command to the beginning of each line.

That way you delete the files bottom up. Even with the extra image activations, this will usually be MUCH faster than a top down delete.

Re: High Interrupt State

Jon Pinkley — Wed, 21 Mar 2007 22:49:44 GMT

First, if possible, do the delete from the node that has a direct attachment to the device, instead of deleting from the node that is being MSCP served. That will help.

Since you have HSG80, you may already be "directly connected", but as suggested by others, it is still possible that you are using an MSCP served connection, if there was a loss of all direct paths to the disk.

If deleting all files in the directory, and it is ok to delete the directory itself, consider using DFU from the freeware disk.

It is much more efficient than deleting the files using delete [somedir]*.*;*.

Can you provide the output of

$ show device/full

of the device containing the directory you are deleting files from?

Jon

Re: High Interrupt State

Martin Hughes — Wed, 21 Mar 2007 23:21:05 GMT

> It is a MSCP Served disk.

Just to confirm, can you do this on the node where you ran the delete and post the results;

$ SHOW DEVICE/FUL

If the disk is MSCP served then try and do your I/O intensive tasks on the node with direct access.

Re: High Interrupt State

Volker Halle — Thu, 22 Mar 2007 01:40:09 GMT

Hi,

the fact that 'the disk is MSCP served' does not provide the information required. The question should be:

Are you using the MSCP path to the disk for your delete operation ?

$ SHOW DEV/FULL $1$DGAn: issued from the node where your DELETE command is being executed, will definitely answer that question. It should show the local nodename in parenthesis after the device name !

Is OpenVMS shadowing involved (DSA device) ?
How about remote locking activity (MONI DLOCK) ?

Volker.

Re: High Interrupt State

Andy Bustamante — Thu, 22 Mar 2007 19:19:07 GMT

What ECOs do you have loaded. I recall there being an (fibre_scsi ?) eco that reduced interrupt time on fibre channel storage.

Andy

Re: High Interrupt State

rison so_1 — Sun, 25 Mar 2007 18:14:30 GMT

Attached Show device detail.

and following patch were installed at Jun 06:

DEC AXPVMS VMS732_SYS V10.0
DEC AXPVMS VMS732_UPDATE V6.0
DEC AXPVMS VMS732_PCSI V3.0

Re: High Interrupt State

Hoff — Sun, 25 Mar 2007 20:59:32 GMT

You can load up the most current UPDATE V11.0 kit:

ftp://ftp.itrc.hp.com/openvms_patches/alpha/V7.3-2/VMS732_UPDATE-V1100.txt
ftp://ftp.itrc.hp.com/openvms_patches/alpha/V7.3-2/VMS732_UPDATE-V1100.ZIPEXE

and try the DELETE again, but realize that the performance of a 2000 block directory DELETE will be comparatively poor.

The obvious ways around that behavior involves DFU or any of the various reverse-order DELETE tools, or to use smaller directories.

There is no fix for this behavior, if it is the directory processing that is at the core here.

Stephen Hoffman
HoffmanLabs

Re: High Interrupt State

rison so_1 — Sun, 25 Mar 2007 21:04:17 GMT

Thank you all for your help.
Useful information were provided.

Re: High Interrupt State

Sebastian Bazley — Mon, 26 Mar 2007 07:53:15 GMT

Just to add: if you don't want to use DFU, then BACKUP may be quicker than delete:

BACKUP *.*;* NL:dummy.sav/save/delete