Another holiday weekend shot ...

[Tom Danzig]

While at my daughter's out of town soccer tournament, get a call from the boss at 5:00 PM Saturday saying that the security guard noticed something burning in the data center during his rounds. Turns out the UPS fried a transformer with bypass mode being unavailable. The manager, being the cautious type, had us shut down everything in the data center just in case of a spike or power interuption. Got home aroung 12:45AM.

Next day parts are flown in and UPS is repaired. Left second day of soccer tournament early to go back to work at 8:00PM and power everything up. Old H40 NFS server won't boot. Turns out the root mirror sh!t the bed and was hanging the bus. Spend several hours trying to figure out what's wrong. Finally, had to open up server and disconnect the faulty drive to get box to boot. Got home aroung 1:30 AM.

Needless to say the holiday weekend stunk. Just needed to do a little ranting to some folks who would understand. Hope your long weekend was better than mine!

[Pete Randall]

Tom,

Well, you've got my sympathy! That just about takes the cake. All I had to deal with was rain washing out any chance of doing anything of interest during my long weekend. You've made me feel much better now, though!

Hope you have a better week.


Pete

[Robert-Jan Goossens]

Hi Tom,

I to have had some disaster weekends in the past. Needless to say I sympathize with you.

Friends tell me that I should have learned for a real JOB, we know better
...((:o))

I hope next weekend will be better.

Friendly salute,

Robert-Jan.

[Nick Wickens]

Bad luck but we've all been there at some point.

Just the one call for me this weekend - My own daft fault because I had taken a backup out of the system but forgotten to remove the alert saying that the backup had not started.

In the words of MP "What A Silly Bunt".

Still if it makes you happy we were up all night on Thursday fixing a failed DNS controller because 2 disks in a RAID 5 group on our LSI Logic SAN (saaaay no more !) went ping within 30 minutes of each other whilst the first was recovering to the hot swap.

[Nick Wickens]

Bad luck but we've all been there at some point.

Just the one call for me this weekend - My own daft fault because I had taken a backup out of the system but forgotten to remove the alert saying that the backup had not started.

In the words of MP "What A Silly Bunt".

Still if it makes you happy we were up all night on Thursday fixing a failed DNS controller because 2 disks in a RAID 5 group on our LSI Logic SAN (saaaay no more !) went ping within 30 minutes of each other whilst the first was recovering to the hot swap.

[Massimo Bianchi]

Fully with you.

It was only one month ago when one of my customer said: we got and ora-600, and alert is full of strange messagess.

It was thursday at 9:00 o'clock.

Good start...

I got home at 15:00 ... but of Friday, with a restore and hate for pizza and coca, and a lot of sleep...

Massimo

[Bryan D. Quinn]

Hey Tom,

I feel for you. I have been there. In fact, at my old employer I worked a long second shift on a Friday night to train a new operator. I left the facility at about 3am, got home about 3:45am. About the time I hit the bed the beeper went off. I called in, the operator (being new) was frantic. She said all of the lights have gone out in the computer room, but all the systems seemed to be running but there were some alarms. I said, that is fine everything has switched over to UPS for some reason. I ran her through the procedure on who to contact to find out what happened with the power. Fifteen minutes later she beeps me back, she is just about crying at this point. She tells me that everything is down, all the lights are completely off and the data center is completely black and quiet. I got back out to the facility at around 4:45am. UPS had failed and everything took a dive, HP-UX servers, one mainframe, a handful of System 36's and an HP1000. It was a real mess, didn't back home again until around 3pm the following day.
That was the best weekend that I have had since I have been in IT. I base all my other sucky weekends around that one!

Hope next weekend is a breeze for you!

-Bryan

[Nick Wickens]

I thought I replied to this - Its probably got posted some where weird !

Sympathies extended - All I got this weekend was an alert because I forgot to cancel the alert telling me that a backup had not started. Fortunatly it was me that took it off the schedule. "What a Silly Bunt".

If it makes you happy though we had an all nighter last Thursday when a RAID 5 disk failed in our LSI Logic SAN (Saaaay no more) followed by a second in the same vol group before the first had recovered. Took our DNS service out (That was when we found that someone decided that putting both DNS servers on the same volume group was a "good idea") but had the added advantage of finally convincing management that booting servers off of this SAN is not a good idea after all (Like we said to start with ! - Why do management always trust Sales people ?.)

[John Meissner]

Tom - my condolences.... that really sucks. To think I was complaining about it raining all weekend.... It could have been soooo much worse. Anyway I'm sorry your weekend was ruined. Hopefully you get some comp days or overtime for your efforts.

[Nick Wickens]

Somethings going on here "DAN HELP !". I am not getting the "thank you for your valued contribution" type message and early refreshes don't show my reply.

So apologies for anyone who is thinking "what a twit". (That goes anytime of course).