hp-ux systems administration best practice thread

[Danny Ramos_1]

hi guys,

am just a novice system administrator and would like to gather expert's approach on system administration. may i then request you to post a brief and concise insight in managing your rather complicated system. Points will be rewarded accordingly. Thanks.

more power,

danny

[harry d brown jr · Accepted solution]

My approach is two fold:

(1) networking - build strong relationships with the following groups:
..(a) your vendors
..(b) your developers - or third party vendors
..(c) your networking staff
..(d) your telecommunications people
..(e) your facilities people
..(f) your electricians
..(g) your business units
..(h) your "clients"
(2) walk loudly and carry a big stick:
..(a) don't give QUICK anwsers to issues unless you are sure of the cause and how to correct it. Take time and get it right the FIRST time.
..(b) you earn respect and respect the job others do!
..(c) don't say "that's impossible", because few things are!
..(d) get excited and use that energy to learn something new
..(e) learn to say NO, because when someone say's they need to open some port up on an internet server that could compromise your companies reputation, then put your foot down and say NO, unless you get an officer of the company to authorize such IN WRITING. I guarantee you won't be seeing a lot of memo's from officers asking you to destroy your companies data and integrity, unless of course you are Arthur Andersen and work at Enron.
..(f) most importantly, have FUN!

live free or die
harry

[Michael Tully]

Have a look at these postings:

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0xf74a49c5ae73d4118fef0090279cd0f9,00.html
http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x9804a22d6d27d5118fef0090279cd0f9,00.html
http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x3f6ffd3f91d3d5118ff40090279cd0f9,00.html

[John Poff]

Hi Danny,

You asked a good question. I thought about it for a few minutes and
here is a short list of what I came up with.

1. Learn something everyday. Reading to this forum daily will do it for you.
Read the trade magazines. Read the manuals. Read the man pages for commands.
You really want to learn something? Do an 'ls /usr/bin' and print the list, and
then start reading the man pages one by one in your spare time. Waiting on something
to print or a phone call? Read a man page. Learn where to find the answers and
you won't have to memorize everything.

2. Learn to write scripts. Shell scripts. Perl. Awk. Learn what you are most
comfortable with, but learn enough of all of them to make your way around.
Look at scripts other people have written. Use the really good things you see.
You'll need this skill for the next thing.

3. Automate when you can. If you find that you need to check something on a system
everyday, write a script to check it for you and have it e-mail you just when
something is wrong. Have it log everything the rest of the time in case something
blows up and you don't get the e-mail.

4. Document what you do. Not only in your scripts, but in your work. E-mail your
boss and your co-workers when you make a change on a system. Keep people informed
about what you are doing.

5. Don't get to be a bigot of one brand of hardware, software, or operating system.
If you stick around for a few years, chances are you'll work on several different
brands of everything. Nobody has the best of everything, and you can learn something
from other flavors of operating systems. Be flexible.

6. Be responsible. If you do something wrong, admit it. I've made some pretty big
mistakes before and I've survived some really stupid things just by standing up
and saying, "I did it. It was my fault. It won't happen again."

7. Keep your cool and have fun. If you live to be really old you will look back on
these days as the time of your life. I'd prefer to remember all the fun I'm having
instead of grumbling about work/co-workers/bosses/business/companies. Get over it
and have fun.


JP

[George Petrides_1]

Danny,
I will also stress out organization, consistency and always keep in mind growth. I have been in large datacenters where the first UNIX machine came and people thought that was it, one machine and we are all set. Before you knew it, a year later there were 50 UNIX machines around. Keep things organized and setup things in a way that other people can work on them (naming conventions, well written scripts), you might say that a cryptic setup means job security but it will also limit your growth. Keep your systems clean (file systems don't let them fill up and then get a call at 2AM), learn a bit about the databases and applications so you don't always need to call the DBA just to help you stop and start the database. Try and document as much as you can regarding your setup (this can be helpful in an emergency when a machine is down and you have absolutely no idea what the HW addresses and configuration is). Write down standards, you can make the rules and other people will respect them and follow them. If you name servers after cities in California or Islands in the carribean, then good luck having other people remember which one is which. If you put 1000 scripts in one directory, then wish other people who need to use these scripts good luck. As the other postings said, try and learn really good shell (or pearl) scripting and automate as much as you can. Help the developers and the DBAs, many of them will appreciate it and it will help your reputation in the company and most important make your life easier. And most important try to enjoy it and have fun!
George

[John Dvorchak]

Danny, all of those answers are great ones. I would like to share my views on a couple of points mentioned by the esteemed group that answered your question.

1. naming convention. Here is what I have used successfully for years:
Make the name not cryptic but not too simple. As pointed out earlier you can name them after cities, Islands etc. but the name doesn't mean much. I like to use a series of letters and numbers that mean something to everyone. Take the name CHPROD1. The first letter is the city it is located in "C"incinati. The second letter is the manufacturer "H"ewlet Packard. Prod is a production server and 1 is the first one online. So the name tells me it is in Cincy, it is an HP and it is a production server. Could be SSDEV3 which would be in St. Louis, Sun and the 3rd developement server. Play with it and arrive at your own naming convention. Could be floor numbers or building sites and application servers. Refrain from a St. Louis, HP, in the IT department.

2. Learn something every day. Get books on the O/S and hardware, networking, Sysadmin etc. And don't forget that until you are an expert use the forum just as you have done here. You no know that answers are quick to show up and accurate.

3. Don't beat your self up if you don't know what went wrong and how to fix it. Don't panic, take a breath and start thinking not doing. After a while you will find that you don't have to know all of the answers, a truely smart Admin knows where to find the answers.

Good luck and

[Elif Gius]

* it would be good if you have patience; somtimes things won't does what you want
* don't be panic;
somtimes systems crashes,cluster wents done, but in such a situatio it is the best to be "cool" at first
* it would be good that you are a little bit organizational;
that saves your head and you won't get confused
* it will help you if you can laugh and have a little of fun



+++ don't drink too much coffee; I've seen colleagus who were getting mad and too much coffee isn't the best for your stomach

[A. Clay Stephenson]

The best piece of advice that I can give you is: "Get yourself a SANDBOX." This is a machine that is even lower than a test environment and it is where you first apply new OS releases, patches, new database patches, ... AND where you learn and play. It is the perfect place to learn how to use Ignite and what the effects of terrible tuning tuning can be. A box like an old D or K on the used market will work just fine and believe it or not, it is a very wise investment. This is one of the main techniques for approaching what should be one of your most important production goals - zero unplanned downtime.

[Dave Unverhau_1]

Danny,

NEVER turn down an opportunity for any decent training.

Keep your options open (good planning helps here).

Get a bound notebook (like a school composition book) and keep it with you. Use it whenever you learn something new. I got that one from one of the "masters" here. You may find it in another thread (my apologies), but it bears repeating.

Document your procedures and activities carefully.

Don't take yourself too seriously. (My wife keeps reminding me of that.)

Remember your family and don't let them suffer on the "back burner" ...keep in mind why you're bothering to work in the first place. (My wife keeps reminding me of thet one, too!)

Good Luck!

Dave

[Wodisch]

Hi Danny,

maybe it doesn't sound that much of a technical advise, but:
1) read everything, as soon as you get it
2) learn whatever you can, as soon as you get the chance to do so
3) try/experiment whatever you cannot answer with the experience from the first two points as soon as possible

Actually this works for me for over 20 years, now. And everytime I turned down a chance to learn/read/try, just the very next day I had to be sorry about that, as it would have saved me a lot of trouble by then...

Ad 1: books, magazines, mailing-lists, news-groups, web-sites (liek the forums ;-)
Ad 2: trainings/workshops, *older* co-workers, even bosses
Ad 3: as Clay wrote: get a sandbox. Create Ignite/UX tapes. Learn to use them. Use them. (Solaris: JumpStart, AIX: mksysb, Windows: Ghost, Linux: well - "mondo" maybe)
With a safety net like that, you can sleep much better.

Explain things to others - afterwards you'll know those topics way better than before!

FWIW,
Wodisch

[Nag Varma]

Hi Danny,
The first place to start I would say is
to learn and know as much as possible about
you existing environment....like..

=> what hardware you have
=> which applications reside on what boxes
=> who are the application contacts
=> whats the contact escalation procedures
=> Try to see if you can get a test box where
you can try commands before executing on
the production env..
=> How the backup is performed ,, like what
software is being used , if this falls
under your area..
=> when system crashes, what are the places
to look ...just couple of minutes ago there
was a thread regarding this..
=> how to do a system recovery
=> Try to understand the concepts of LVM,
network setup etc..
These are some of the things I can think
Hope might help..
thanks
Nag Varma

[David Bell_1]

Danny,

All of the suggestions you see before you are good ones. In the form of documentation (for HP), I prefer to use the "Gold Book" approach and keep it up to date. The Gold Book was a book that HP used to provide that documented all I/O, printers, disks, software, service calls, etc. In addition, I added all the LVM information I could create, ioscans, network information, user information, application information, etc. However, far and aaway the best tool I use is Visio along with the templates provided at

http://www.sisp.hp.com/SISP/LoginFrame.tcl?area=stencils&doc=info/VisioStencils

These are great for making a living picture of your computer room. I post this to the admin web page as pdf's so that admins can drill down to details and see io addresses, etc. These books can be a bit overwhelming if you're in a large data center. However, I try to make them a priority upon the arrival of a new system.

HTH,

Dave

[Steven E. Protter]

Purchase a book called HP Certified by Rafeeq Ur Rehman.

There are mistakes. The latest version is missing he answers to the practice test.

You don't need to take the test. This book contains all of the basics you need to succeed technically as a Systems Administrator.

I got it after browsing through it and stumbling upon answers to serveral issues I was working on.

The other advice here is also excellent. Follow it and improve your skills and you will do well.

Steve

[GregD at VisioCafe.com]

Hi, well that link is an outdated server that hasn't been updated since last October.

There is new server though,

Check out the home of all HP owned Visio Stencils.

http://www.visiocafe.com

Greg Drinkwater
VSD Grafx Inc.
visio-cafe@hp.com

[Tim Sanko]

Danny,

The first and most important thing for success as an admin is to know when to say "I don't know, but I will investigate and find out." It is often a truth, and it allows you the time to find the right answer.

Also, you can't know it all. So know everyone. Everyone has skills at one thing or another.

Be quick to listen, and slow to speak. You can't be too quiet while someone is describing a problem. Use the mute button on your phone. Especially when you describe a user accurately as a "surviving brain donor" or something more even exotic.

I know too much and often open my mouth too soon. Even when I am right, some believe I am rash to state "it looks to me as if..."

Tim

[John Meissner]

a very good question Danny,
I'll keep it as short as I can :)

here are some things in no particular order

- learn scripting. I use this ALL the time. two good books are "Korn Shell Programming by example, O'Brien, Pitts" and "sed & awk, Dougherty & Robbins"
both are excellent and have helped me significantly.

- Learn from those you work with. When I started administrating Unix systems about a year and a half ago I didn't know much at all... I still have a lot to learn. Some of my co-workers have been doing this for 10+ years and I find that I can learn a great deal from them.

- done give out your password. This can save you a lot of headaches

- Documentation. Document everytyhing... if someone else can read ... then you might not get called out in the middle of the night.

- Pass the buck. Not in the slacker context... but rather install a tool such as Sudo or Service Control Manager and let a lower tier support team do some of your more meanial work such as account creation.

- participate in the ITRC forums!!! I can't begin to tell you how much I've learned from the people on here... and I've had several questions answered on here without having to go to HP support for help.

- Install a linux box on your desk. I would rather have a linux box on my desk than an HP workstation - It's just better. I have all kinds of scripts I've written in bash to monitor my HP-UX environment. Plus chick's dig linux (just kidding).

- Attend training classes periodically to keep up to date on what's going on or new tools and techniques.

- adopt a password policy and stick to it. Many users will not like this but setting a password length and aging policy will enhance security. Make sure the password also consists of alpha and numeric characters.