HPE 9000 and HPE e3000 Servers
1748163 Members
3762 Online
108758 Solutions
New Discussion юеВ

V-2600 CPU 0 maxed out

 
Bryan D. Quinn
Respected Contributor

V-2600 CPU 0 maxed out

We have a V-2600 running SAP/Oracle. On Monday morning the OS hung, meaning I could put in my user id but it would not come back and ask for my password. It was just frozen. I could log into SAP and it seemed to be running fine. The OS was just frozen. My first thought was that something ate up the memory in the box (we have 28GB).

This scenario is not a new one, in the seven years I have been here this exact scenario has happened about five times. Always just after midnight on Sunday night/Monday morning, with the exception of one time when it happened at 8am on Monday morning. We reboot all our UNIX servers every Sunday night at 7:30pm, so this is happening just a few hours after a reboot.

Ok, this time when it happened we did some snooping from inside of SAP. Looking at some OS information and noticed that the following process had CPU 0 pegged out.

/opt/java/bin/../bin/PA_RISC/native_threads/jre -lazyswap -n

It seems to me that one of the previous times that this happened we saw another process that was eating a CPU, whether it was CPU 0 I do not recall.

So, my question is for all you that might have more knowledge of the V's and it's architecture, would it be possible that if you pegged out CPU 0 it would hang the box? Is there some magic about CPU 0 that I am not aware of?

Of course this is a long shot, that process might not have anything to do with the system hanging. So, based on that if anybody sees anything that I might be missing in this scenario, please let me know.

Any help would be greatly GREATLY appreciated.

Thanks,
-Bryan
12 REPLIES 12
Torsten.
Acclaimed Contributor

Re: V-2600 CPU 0 maxed out

The very first idea I have if I read about java in combination with an old server is: patches. IMHO you should check (e.g. a patch session on ITRC or via SWA - software assistant) if your system needs patches.

Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
Bryan D. Quinn
Respected Contributor

Re: V-2600 CPU 0 maxed out

Hello Torsten,

I think we are up to date there...or should I say as up to date as we can get on an old box. I will definitely check that out.

I will respond with what I found.

thanks,
-Bryan
Bill Hassell
Honored Contributor

Re: V-2600 CPU 0 maxed out

Each CPU in HP-UX is equal except that CPU 0 is usually the Monarch processor and becomes the control center for other CPUs when they run kernel code. There is nothing magic about using 100% of any CPU -- that will not hang the system. But as mentioned, patches should be up to date *and* Java should also be updated. Also check on swap space (swapinfo -tam) to make sure you are not reaching the limits. 28GB is a lot of space but Java and SAP and Oracle can eat this space very rapidly if not constrained. Also check your syslog for any problems.


Bill Hassell, sysadmin
Bryan D. Quinn
Respected Contributor

Re: V-2600 CPU 0 maxed out

Torsten and Bill I appreciate your replies.

Once again, this morning we had the same problem. And once again the process eating up the CPU was:

/opt/java/bin/../bin/PA_RISC/native_threads/jre -lazyswap -n

I am not sure what Java is used for, other than the fact that we had to install it to do the Oracle install.

Does anyone know what purpose this process would serve on an Oracle/SAP box, other than to do an install or upgrade?

Thanks,
-Bryan
Torsten.
Acclaimed Contributor

Re: V-2600 CPU 0 maxed out

No idea what it is used for, but I still think you should check for patches:

http://h18012.www1.hp.com/java/patches/index.html

Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
Bryan D. Quinn
Respected Contributor

Re: V-2600 CPU 0 maxed out

Thanks Torsten

I will check the patches again. Make sure I did not miss something.

Thanks,

-Bryan
Bryan D. Quinn
Respected Contributor

Re: V-2600 CPU 0 maxed out

Further investigating this problem, I found that the parent process for the above mentioned process is:

/bin/ksh ./emShell.ksh EM /opt/SiteNetML em.cfg

Which is our Liebert monitoring utility. Which I know nothing about because I did not install it, it was installed before my time.
Chris Vail
Honored Contributor

Re: V-2600 CPU 0 maxed out

I also ran an 2600 for a couple of years and experienced the same problem. However, I left a 'back door' in it for just such an emergency. The back door in this case was a .rhosts file. I found that even a thoroughly hung system would frequently respond to an RPC request.

When the system hung, I could login to another system and force a reboot:
remsh $HOST "shutdown -ry 0"

Please note that at one point I typed
remsh $HOST|shutdown -ry 0

This was one of the biggest fat-finger events I'd ever done.
Bryan D. Quinn
Respected Contributor

Re: V-2600 CPU 0 maxed out

Thanks Chris for the information.

Did you ever resolve the issue?

Did you figure out what caused the process to peg out like that?

Thanks,
-Bryan