You know you have got it wrong when:-

[Trond Haugen]

I went to a IT-fair SOME year ago where the exhibitors learned the 2nd morning that if they had wanted 24hr power they should have ordered it.....

Regards,
Trond

[Justo Exposito]

When your boss is wating for you at 8:00 AM

[Bill McNAMARA_1]

you've got pieces left over...

;)

Later,
Bill

[Deepak Extross]

your new SysAdmin wears a MCP t-shirt.

[Alexander M. Ermes]

..you run a sql-script on the fresh created database and the production guys suddenly call you because of missing customer data.
( Took 8 hrs to restore everything with a recover until time on an Oracle database ).
Rgds
Alexander M. Ermes

[John Strang]

you suddenly find you've got a lot more free disk space :-)

and it's nice to know I'm not the only one who has shut down the production server by mistake! And I too now always insert the hostname in the prompt.

[James Beamish-White]

Your nifty script that cleans up scrap files ends up rm -R /* cos someone changed the /etc/profile. Doh!

Ah the hard education of the beginner.... I think that was one of my 24 hour stints from memory.

James

[Pete Devlin]

You realise too late that shutdown 0 now on a remotely managed machine (without modem access) 110 miles away ain't gonna leave you in a position to do much. Done that & had to be VERY nice to somebody not on call & who was nothing to do with the server to go & get it back to init 3 - wouldn't have been too bad except it was millenium eve........

[Emiel van Grinsven]

you created a copy of the passwd file, edited it and placed back without checking the permissions. (and some loose cannon just changed the umask without telling anybody)
Then you get a call saying the application is not working properly (internet banking application) ...

[Rita C Workman]

..when the guy watching behind you says " ..is that really what you were going to do?" [and of course it wasn't I meant to do]

..the UNIX trainee informs you that he has just rebooted the Production server, cause the DBA told him to. [Phone ringing...it's the Director]

..nobody can log into the database one day! [and you remember cleaning up and making the hostfile a little neater - but seems you put an IP in the wrong place in the hostfile, so tnsnames ain't a-workin now - oops!]

..you change root passwords and put the new info in your purse...change purses and get called on a Saturday evening 'cause there's a problem. [..your at a wedding and miles from home]

Yep ! All happened ! And like everyone else I have rebooted the wrong box too.. fortunately it wasn't one of the production boxes..but I've really messed up some folks when they were running tests.

Rgrds,
Rita

[Rita C Workman]

You you don't triple check EVERYTHING the previous UNIX Admin did..

..you notice that they extended the operating system (vg00) to the external disk array to get some more space. And then the fiber connection to the disk array goes down, the primary boot goes bad and you find that the mirror disk is not truly bootable! It was not a good day..!

[...NO I DIDN'T DO THIS! I inherited this mess. Couldn't get mgt. to purchase some disks to go into the Jamaica to try and clean this up (cause the box works fine...why do we need to buy those little disks..). Well when the fiber card & boot disk went bad and the guy had apparently set up the mirror boot disk wrong (my fault I hadn't checked the mirror for bootable), well they found out! It took days to fix this mess.]


Always check EVERYTHING !
Rgrds,
Rita

[harry d brown jr]

when...

You get a call from the operators that the system isn't responding to anything...you can't login so you boot the system into single user mode and NOTHING works except for the built in commands like "cat *" to list files and theres nothing being returned. You work the morning restoring the system (fortunately the backup was good).

Then the next morning the same thing happens!!!

So we know it's got to be a cron job, but what the hell do we look for? After a few minutes of thought, you remember deleting a lot of files and directories from the /tmp directory. HUMMMMMMM!

So we go looking for cron jobs that are looking at /tmp, and lo and behold we find a THIRD party software (uniplex - about 12 years ago) provider that has a cron that cleans up it's own /tmp area, but unfortunately the IDIOT that wrote the script must have been MCSE certified.

The ROOT CRON script, at 5am, attempted to "cd" to the NON-existant /tmp/whatever (fake name because I can't remember - must be old age) directory, then proceeded to do a "rm -rf *". Well if the /tmp/whatever directory doesn't exist, then the "cd" failed, leaving the CRON process in "/", where it proceeded to REMOVE EVERY SINGLE file and directory. And the 'turds didn't pay me to fix their script!

Lesson: Don't trust nobody, especially third party software vendors!

live free or die
harry

[John Bolene]

You get a call from Info Security to reset their password on their machine and they think that you have root's password on that machine.

Now why would they think that, it is their machine and they are totally responsible for it?

Had to drive to work, boot in single user mode and reset the root password for them, that they should have known.

[Jack Werner]

An Administrator inadverntly removed /usr/ccs(for God knows what reason!) and every "C" command on the crippled machine errors out. We had no recourse but to reinstall the OS from scratch, since we also had no bootable copy of the OS!!

[Darrell Allen]

unknown to you, the application vendor dials in to clean up a couple of items in the database and suddenly production stops. (DB was extremely clean by the time I stopped her. At least management realized it really wasn't a good idea to allow unrestricted 3rd-party access to our systems.)

your 3rd-party vendor begins to install a new application on the backup system in a ServiceGuard cluster and now the production machine will no longer boot. (That shared disk was the data disk, right? Wrong!)

[Scott Van Kalken]

You walk into the computer room and smell nothing but electrical burning... it begins to make sense why your machine is in the middle of a core dump (really happened to me).

[A. Clay Stephenson]

When you say to your HAL-9000 computer, "Open the pod bay doors, HAL." and it replies,
"I'm sorry Dave, I'm afraid I can't do that."

[Sonny Chee]

True story, I inherited this server, I run a userdel on a userID, I noticed it was taking
a long time. Then the users starting calling
and said they can't get into the application.
I found out that the user's home directory is
the same as the application's. When the userdel
ran, it removed the application directory. It took 3 hours to restore the application.


Sonny

[Mark Fenton]

you "didn't have time" to run the fix in test environment first before wiping out half your production database.

(Fortunately we did have a good backup, and so only lost six hours' production.)

You clean /tmp and discover that a lame programmer was "hiding" a license lock file there. (three phone calls, one hour, and a lot of pissed-off people later...)

[Frank Slootweg]

[1980's story:]

You tell this customer to follow the RGB cable from the monitor to the computer to the panel with thumb-screws and tell the product number on the panel (because it will tell you the type of graphics card, which will tell you what to set TERM to).

After many yes-no rounds, the customer still insists that it isn't a small panel with thumb-screws, but a large panel with normal screws, and that the product number you are looking for is not on there.

When you are about to have the customer committed, you find out that "uname -m" was 'lying' to you and the customer has a different system than you think he has. Oops!

[Of course this did not (? :-)) happen to me.]

[Gnananandhan]

After giving a rm -rf from the root directory, Thinking I am in the /tmp directory, It crashed of the whole system. Caught with panic and lost 1 day business due to this.


Gnana A.

[Steve Steel]

When enable printer gives
Printer is already accepting requests

Happened to me.

Someone disabled printers. removed enable and linked it to accept.


You know you have it wrong when
/etc/motd is

resistance is futile
You will be assimilated

[Trond Haugen]

you trip in the UPS powercord and the rack next to it suddenly goes quiet...

Yes, that was me,
Trond

[Martin Johnson]

A vendor has come in to install a demo version of their software. I was called to deal with a production problem. After dealing with the first problem, I started getting complaints that users could not access a sysbase database.

It turns out the vendor needed space for the install. Using SAM, the vendor took a "unused" logical volume, created a file system on it, mounted it and loaded their software on it. Unfortunately, that "unused" logical volume was one of the raw partitions being used by sysbase.

I had them de-install their software and escorted them to the door.

Marty

[Carlos Fernandez Riera]

Every thing is running ok .... It is enough to suspect.

You notice that have forgotten your cell phone after 2 hours of driving...