Linux server reboot every two month

[jack Hu_1]

Dear Sir:
I have HP Proliant BL460C server with Linux Red Hat Enterprise Linux ES release 4 (Nahant Update 4)
2.6.9-42.ELsmp #1 SMP Wed Jul 12 23:32:02 EDT 2006 x86_64 x86_64 x86_64 GNU/Linux
I found that it will reboot every two month.
Another server HP D380G5 with the same linux version.
They all have the same problem that server will reboot every one or two month.
Is there any suggestion or any experience about these kind of event.
thanks for your support.
Jack

[Ivan Ferreira]

More information is needed, the date and time of reboot, and any messages registered on /var/log/messages on that date.

Also, identify if it was caused by ASM for example because of an evironmental problem.

[Marco A.]

Hi,

Don't you have any kind of cron job running performing that action? or any other job that could cause this due to a overflow, crash, etc ?

Regards,

Marco,

[Taifur]

HI Jack,

This issue may be related with any hardware parts such as RAM, CPU.Pls check reseating RAM,CPU and check shutdown.log, syslog.


Rgds//
Taifur

[jack Hu_1]

Dear Sir:
The reboot time is not regular.And there is no cronjob for re-start server.I don't see any error message on the message file at the reboot time.I just wonder that why the same linux version of the two server will have the same problem of auto-rebooting.

[avizen9]

is there any cronjob running for it
can you post crontab -l output, thanks

[Huc_1]

I do not have your hardware software set up so could not tell you if this is also true for other system of the same "type/setup", and you may already have coved most of what I would look for but I would double check the following thinks on each system, try and narrow down your search and only record fact,before you jump to conclusions

at around crash time what do files in /var/log/* log on each system, make a compare list

Is there a pattern 'only on week days' never during week end ?... only on week end... never at night etc ..

also I would not limit myself to the system, (hardware and software) but would have a look at all surrounding environment, like perhaps the are there any other electrical appliance that would or could only run every 1,2 months and cause theses problems.

The above reply is vague, but we realy need more narrowed down informations before whe can get more specific...,

Keep at it, and report here, bets are we will be able to help.

Enjoy life.

Jean-Pierre Huc

[Ivan Krastev]

Do you have any PSP (Proliant Support Pack) or part of it installed? Some of the older version have problems and server hang/reboot.

regards,
ivan

[Tim Towers]

Firstly I would look in /var/log/messages to see if you can see shutdown messages which would indicate that the reboot was a controlled event.

If everything seems normal and the restart has no warning then you could install kdump to see what state the kernel was in when it rebooted.

We have systems running RHES4 on x86_64 and they do not reboot without good cause.

[Nuwan Alwis]

Hi Jack,
I have installed same OS on same Blade server and i havent came across such a situation. The only thing which i dont do is installing the PSP unless if it is really necessary. wel this could happening due to a application malfunction, faulty hardware such as HBA or NIC.
if you could send us the /var/log/messages
we can find a clue for the course of the reboot. (note us when the reboot occur and time)
also make sure about the cooling system of both servers.

Run a smart-start offline diagnostic with about 8-10 iterations excluding disks. this will give you a confirmation about your hardware.

Good Luck..!

[macosta]

Jack,

See if you have PSP installed. If so, please visit the following for details:
https://bugzilla.redhat.com/show_bug.cgi?id=243657

Essentially, the mainline kernel had a bug in it that leaked memory on x64 systems with >4G of memory when cciss_ioctl() was called (which is called by the hpasm services.)

It is fixed in RHEL4 update 4, and in kernels derived from 2.6.10 of the vanilla kernel.org source.

This may not be your issue, but I did notice this a while back, and it did affect systems of mine.

[jack Hu_1]

Thanks for all your quickly answer my question.I attach the message file below at the reboot date.
About the HP PSP I know is version 8.1 .
I also check that another linux maybe will have the issue of PSP , 'cause that the PSP version is older that 8.1 . And it will reboot every one or two month. And the linux version is older that 2.6.9 .
And someone say maybe it's a H/W issue. That's true on one of my server(not the two server I ask ). After change M/B and re-install OS as 2.6.9-42 , the server is ok now.(never reboot again).

So, I will go to check PSP also the bug of memory issue.(I found that one server's memory usage will grow up until reboot server).
All the suggestions I will work with vendor to double check the root cause.

If someone found some error on the message file,please let me know.
thanks for all your help.
Jack

[macosta]

Jack, the -42 kernel release is the latest as of RHEL4 up4, so if that kernel does not display the issue, my previous comments may not be relevant. Again, this is only the case if you have an affected kernel and PSP installed and activated.

If you have support with HP, I'd suggest you contact them, where they could review your PSP, kernel, hardware, etc.

[Nuwan Alwis]

Hi Jack,
Can you send us any previous message logs because this only shows startup events.

Any way did you check the HBA cards and SFP cables?