topic Re: How often do you reboot....and why? in Operating System - HP-UX

How often do you reboot....and why?

Ted Flanders — Sat, 24 Feb 2001 17:22:33 GMT

I run a single HP9000 K220 w/HPUX 10.20. I reboot my box bi-weekly. I have heard to sides to doing this. HP says absolutely do not do this, your hardware might fail on boot up. Frankly, thats fine with me, I pay 16K per year for a service contract and all parts and labor is covered. We have used this contract once in the last 2 years and got charged an extra $1,800 for a "special" charge.
My software people tell me to reboot weekly. The other sys admin here tells me bi-weekly and my boss tells me to "find out".
What does everyone here do? And why?
Thanks.

Re: How often do you reboot....and why?

Ajitkumar Rane — Sat, 24 Feb 2001 17:41:56 GMT

We do not reboot the servers unless there is a reason for doing it, like patches, upgrade etc etc, so the servers do not get rebooted for months together.
I remember reading in the sys admin manuals, it is recomended that you should unmount your filesystems and run file system checks to keep your filesystem intact and know of any errors before they suddenly shoot up at production hours.
Sometime back I was in a shop where the server were rebooted monthly, as they had a application which caused memory leaks.
Now there is a difference in robooting the server and shutdown with power cycle.Power cycle can cause hardware failures.

Re: How often do you reboot....and why?

Rita C Workman — Sat, 24 Feb 2001 17:57:41 GMT

I do not reboot that often. Whenever patches (or new bundles) are installed, and those quarterly bundles I only do that maybe twice a year. I reboot whenever there is a serious problem with the system and the only resolution is to try a reboot. I used to try and just do a reboot maybe once a month. But even then folks complained about the downtime. So I keep reboots to a scheduled minimum and only when absolutely no other alternative.

/rcw

Re: How often do you reboot....and why?

Vikas Khator — Sat, 24 Feb 2001 18:50:31 GMT

Hi,

I believe in rebooting servers once a quarter and usually coincide with patch bundle install.

Only other reason I would reboot more often is to fix memory leaks . ( a very common problem in devlopment environment) . However right thing is to get the devlopers to fix the memory leak.

Re: How often do you reboot....and why?

Patrick Wallek — Sat, 24 Feb 2001 20:29:55 GMT

I have one weekend per month where I get the servers to myself with no users. During this time I reboot the machines, run fsck on all filesystem (while in single user mode), make sure I have a good backup of the system with the database down, resize the database files (this is a Unidata database), install patches, and whatever else needs to be done.

This schedule has been working for about 3 years and I generally don't get too many complaints from users.

Re: How often do you reboot....and why?

Patrick Wallek — Sat, 24 Feb 2001 20:32:23 GMT

Just had one of the other responses sink in.....

I do agree with one of the previous responses that said rebooting and powering off are distinctly different. I don't turn power off to the machines unless I absolutely have too. You do run the risk of having hardware failures then.

Re: How often do you reboot....and why?

Rick Garland — Sat, 24 Feb 2001 20:33:57 GMT

You are right, different sides to the issue and I find it depends on your environment. If you do have troubles, like memory leaks or something else, you will need to reboot more often. The software dev folks like to see reboots more often than what is typically necessary.

If you have a system in the prod environment, chances are you will need to reboot less frequently.

And if doing patches, some patches require reboot after installing - can't get around this.

You may be able to run a system for a year or more without rebooting or you may decide to get the SW dev folks to be happy by rebooting more often.

The systems are designed to run without having to reboot and for some apps the systems will do just that. But developers have a habit of introducing issues in their code that only a reboot will clear up. Then get the developers to fix their code.

Re: How often do you reboot....and why?

Bill Hassell — Sat, 24 Feb 2001 23:22:08 GMT

I never reboot except to apply patches or to add new hardware. I ran a 9000/870 as an email server from 1991 to 1999. I had one unscheduled reboot to fix a runaway NFS problem that was later patched. For the first 4-5 years, it ran 24x7 for at least 400-500 days.

Now as mentioned, poorly written programs can use up RAM but that's easily fixed with the kill command. Same with runaway programs that consume CPU and/or LAN bandwidth. There is no need to reboot HP-UX as long as you keep up on patches.

Re: How often do you reboot....and why?

Philip Chan_1 — Sun, 25 Feb 2001 03:52:15 GMT

Yes, rebooting the server could cause damage to the hardware. This is hard to believe but really happened to my site a few weeks ago. We cold booted 5 HP servers and 2 of them resulted in hardware failure (CPU in one server and disk drive in another). I think this is due to voltage fluctuation during cold boot (warm boot should be fine).

Having said that reboot could damage hardware, I still think that rebooting servers periodically are good practice simply because:

1. every restart will force the fsck to run against your disk drive.

2. will clear any zombie processes and ill behaved processes that holding up memory.

3. will ensure your server be able to start after any system change.

4. it doesn't do any harm providing that
- you got a hardware support contract with HP
- your disk drives are mirrored or RAID'ed (or else you'll have to restore data from tape)

Regarding the time, I guess once every month should be fine.

Rgds,
Philip

Re: How often do you reboot....and why?

Ted Flanders — Sun, 25 Feb 2001 04:52:17 GMT

I warm boot, and do not cycle the power. I mainly do this because my uniplex email starts misbehaving. It doesnt notify users when they have mail, etc.

Re: How often do you reboot....and why?

Jitendra_1 — Sun, 25 Feb 2001 05:52:14 GMT

I used to work in a development shop on my previous job. There were number of BEA developers and we used to reboot the boxes every sunday . This used to clear up a lot of memory otherwise held by some stray user programs .
However occasional reboots for patches and maintenance also were normal.

Re: How often do you reboot....and why?

Gram Ludlow — Tue, 27 Feb 2001 17:57:53 GMT

We reboot our servers only when we install patches that specifically require reboots. Hardware failures can and will happen, especially with hard drives. Once a place I worked for lost power and cold-booted a RAID5 disk array attached to a server, and two drives died. We had to restore over 100gig of data from backups.

If your software recommends "rebooting" every week, then I suggest you run a cron job to get the pid, kill it and re-start you software. This could fix memory leaks without killing anything else on the machine, and would be automated enough not to be a hassle. I would leave the servers on as long as possible, however.

Re: How often do you reboot....and why?

Victor BERRIDGE — Tue, 27 Feb 2001 18:14:53 GMT

Hi Ted,
I had a HP 827s running SAS that went on reboot every friday evening supported by HP (...) it worked like that during 7 years without any trouble.
I have a an AIX machine that is rebooted every night since 8 months, I decided this because I couldnt deal with the machine rebooting any time in the day, and am still looking for the bug (This machine controles all our working time and so has about 70 badging modems behind...) so its a workaround...
And the others, it depends if there has been some patchs install that require, or because I have to reboot because some machines are on a same disk subsystem and Ive just done some maintenace or disk upgrade...
so time will vary from once every 3 months to once a year
I had a 822S that wasnt rebooted for 950 days (until HP came and took the machine away...)

All the best
Victor

Re: How often do you reboot....and why?

Mark Vollmers — Tue, 27 Feb 2001 18:23:31 GMT

We normally do not reboot, and did not do so for many months. Recently, we have had to many times, due to a corrupt file (or files?) that have caused the server to crash repeatedly. I was told by our old admin. that the server is stable, and did not need to be rebooted. It might not be a bad idea to do it every now and then(I wonder if out files might not have gotten screwed up if I had rebooted every few months to clear it out).

Re: How often do you reboot....and why?

John Bolene — Tue, 27 Feb 2001 21:36:43 GMT

We reboot the application servers about once a month because Hyperhelp has a memory leak that has never been fixed. The other servers are only rebooted when patches are applied, and the DNS server has been up almost 2 years (it was taken down at that time when we were adding more power islands and took that opportunity to power the building down and replace the main power switch with a larger one.

Re: How often do you reboot....and why?

Karthik_2 — Wed, 28 Feb 2001 00:58:48 GMT

we normally dont reboot production servers until and unless there is a need like patch install or some hardware failure .No proactive reboot is done.But we do reboot (warm boot ) our developement servers on a regular basis due to the memory leaks and frequent patch installs and application installs.

In few situations I have seens the hardware especially hard disks fail after a cold reboot.
It is also a practice that we reboot systems before handing them over to other sys admins or other teams /departments as a part of handover.

Re: How often do you reboot....and why?

Subhas R. Reddy — Thu, 01 Mar 2001 01:13:22 GMT

I work in a large datacenter which runs 24/7/52. We have a Mainframe, a bunch of SUN and HP midranges and a bunch of Intel based servers (NT and NetWare). The only boxes that get rebooted are NT boxes which are rebooted on a regular schedule either automatically or manually. SUN and HP systems never get rebooted except for hardware and /or software upgrades. Mainframe folks do IPL every weekend. I have not seen any problems on SUN and HP systems that can be attributed to systems being up for a long time. Not even on NetWare servers. The ones that need rebooting on a regular basis are NT Boxes only and none else.

Re: How often do you reboot....and why?

Peggy Fong — Thu, 01 Mar 2001 03:44:37 GMT

I worked in a large scale shop with 300+ 9000 series (D,N,T500,T600,H,I,J,...) Like many of the responses we did not reboot the systems since we were 7/24 on most applications. Some were automatically rebooted monthly through cron and some weekly - these were application users. The only reboots I supported were for system changes (software, firmware, hardware, etc). And I found that if you had a primary server or servers with backup servers waiting to take over it would be wise to use the backup servers occasionally. In the old days we had Switchover/UX which required a reboot to get to the backup server. (I actually had a primary fail - the backup rebooted as the primary and it had a hardware problem as well - so we were down for 4+hours). With ServiceGuard we really don't reboot we just move the packages from host to host to make sure all hardware is working with the applications. Funny, but all the reboot requests came from our applications, never sysadmin or operations.
Peggy

Re: How often do you reboot....and why?

Steven Sim Kok Leong — Thu, 01 Mar 2001 05:25:53 GMT

Hi,

In my environment, I usually reboot to:
1) remove hung processes ie. kill -KILL has no effect
2) clean up the harmless processes
3) clear memory leaks

Hope this helps. Regards.

Steven Sim Kok Leong
Brainbench MVP for Unix Admin
http://www.brainbench.com

Re: How often do you reboot....and why?

David Mabo_1 — Thu, 01 Mar 2001 06:08:45 GMT

I have a number of servers running HP-UX, plus 2 SCO 5.05 and 5.04, Linix and a number of NT servers. We run databases, web servers, mail servers and development machines. On the Unix boxes, we never reboot unless:

1. Change Hardware.
2. Patch or upgrade OS.
3. Something will not kill, or we have a problem.

This means 1 or 2 times a year, sometimes quarterly. I once looked at a D200 and found it had been up over a year w/o a reboot.

Now Nt is another matter, we seem to reboot some server often, and most monthly.

I also support 2 24x7 customers, and they only reboot when we patch.