5.1b tru64 boot extremely slowly

[Ramazan_3]

we have 5.1b trucluster system on GS320 machines. Also Oracle 9.2.0.4 RAC installed.
Both nodes boot extremely slowly, it takes one hour and there is no errors while booting.
Is there any one who is faced with this problem?

[Han Pilmeyer]

What kind of storage?
How many adapters and LUN's?
Any particular point during the boot where you loose a lot of time?

[Ramazan_3]

storage is EMC DMX.
2 Emulex adapters and cluster boots from SAN.
and we have 11 LUNs.
thanks for your help

[Vladimir Fabecic]

Hello
Once I had simular problem because of vm kernel parameters. Can you post your vm kernel parameters? How much RAM memory do you have?

[Ramazan_3]

I only changed the followings ;
new_wire_method = 0
vm_swap_eager = 0
the other parameters are default values.
memory is 4 GB.
thanks for your help.

[Han Pilmeyer]

GS320 systems can take quite a bit of time. Close to one hour isn't extreme for the largest GS320 configurations (although still a bit on the long side). If you only have 2 HBA's and 11 LUN's total then you don't have a very large configuration. Are you sure you only have 4 GB of memory. I would think that so little memory isn't even possible on a GS320.

Can you post a boot log (preferably with time stamps)?

On V5.1B, new_wire_method should be 1. In fact, starting with PK5 (BL26), we've fixed it at that (and removed the parameter).

[Ramazan_3]

there are 8 QBB in GS320. This one is only one of them. I mean only one QBB, so 4 GB memory is reasonable.
Normally booting ths machine takes maximum 15 minutes. Now it takes so long time.
Actually, the problem is not only booting slowly. After booting the machine, machine runs also very slowly. Top command shows a lot of idle CPU and memory but even "ls" command can not be entered in this situation.
I attached the boot logs with time.
Thanks for your help

[Han Pilmeyer]

Does indeed seem kind of slow. The first indications of slowness seem to be around the start up of LSM (although that's not typically fast eny way). Also the message around cluster root being on a private bus seems strange.

Can you describe your storage configuration in some detail?

Can you do a "drdmgr dskX" for your cluster system disk?

What's the patch level?

[Mark Poeschl_2]

What changed between when your boots took 15 min and now that they take an hour?

I agree with Han - that message about cluster root being on a private bus looks very suspicious.

[Ivan Ferreira]

We had similar problems.

There are two things that could be wrong.

The shm_max parameter in the kernel.

The cluster interconnect (specially for lan interconnect)

You should check the shm_max parameter, the max recomended value is 4G-16M (i guess).

If you are using memory channel interconnect, check the jumpers and the window (see cluster hardware installation and configuration manual).

If you are using lan interconnect, check ethernet speed, duplex, switch, cables, etc.

[Ivan Ferreira]

I forgot to say.

If the problem is the cluster interconnect, when booting a single member (the other turned off), it boots normally.

[Michael Schulte zur Sur]

Mark+Han,
as it is mentioned here, that can be a false message when using LSM.
http://ftp.support.compaq.com.au/pub/patches/dunix/v5.1b/PatchInstallGuide.txt

I was told that in a cluster the swapping should be eager. Is that correct?

greetings,

Michael

[Han Pilmeyer]

You're right. I don't recommend LSM on large systems and especially not on the cluster root disks. So I had not seen that message before.

I'm not aware of any relation of swap_eager to LSM or clustered systems. Should be fine to run in lazy mode even with LSM in clusters.

As you can see in those release notes several LSM performance issues were fixed. That's exactly why I asked what the patch level was. Which still hasn't been answered.