adive needed on creating a system admin team

[someone_4]

Hello ..
I need your thoughts about forming a system administration team. What is the best and more productive way to creat a system admin team. Right now we have about 25 to 30 hpux boxes running 11.0 and about 40 sco unix IVR boxes amd growing. How many people would we need on a team to admin theese many boxes.And what jobs would we be responsible for? Right now the way it is heading is that we have a hardware and a software team. But is this the best way to do it? The hardware team is responsible for running cable and fixxing hardware the sofrware team does everything else that is 11.0 related. I dont think this is the best way to do this. I feel one team should do everything. This is the first unix system admin team this company has ever tried to create. I would like your thoughts on this. Right now the IT director doesnt feel that we need a team that 2 people can do everything. 1 to work on software and 1 to work on hardware. I would like an idea of how you have your system adim teams set up and what duties actually go to the team. All information and thoughts and questions are welcome.

Richard

[Timothy Czarnik · Accepted solution]

Richard,

I would think, in a perfect world, that all members of the team would be able to perform all necessary functions. Splitting up hardware and software seems logical, but because they are so interweaved, especially on a UNIX platform, I would think you'd want everyone to know how the software reacts to the hardware thats present (new disk/tape adds new device files, multiple paths to devices creates multiple device files that point to the same piece of hardware).

Also, unless you are moving servers around constantly, I wouldn't think that there would be enough hardware work for a whole team. Once a server is set up and configured, it pretty much runs by itself. I understand that disks fail and such, but not often enough to keep multiple people busy.

It has helped me considerably in my work that I have been able to do hardware AND software work (installing, troubleshooting, configuring, etc...). I would think that an added benefit to the team as a whole is that it would keep work more interesting, perhaps cutting down on the turnover.

Just my $.02.

-Tim

[Sachin Patel]

Hi Richard,
In my company we have about 50 V,N,K,J and C class server. Around 80 HP-Unix workstation. We have Five unix administrator including supervisor. four are senior administrator except me. We all does Hardware and OS related work.
But mostly if it is Swinstall, configuration, third party sw, DNS, NIS I do that work. One guys does mostly all power related work like how much power require and so on. One guy does mostly internal hardware stuff. one lady does mostly inhouse apps and new systems installation work. But we all know mostly everything. We all runs printer, plotter or network cables.

Now we have 1000 node linux cluster and we have add one more guy with linux experience. We all now works on linux too.

I think it is up to the upper managemet to setup teams.
IN my view we need one team. Jack of all master of one. In case if cable guy is sick we don't want to stop laying cables.

Sachin

[Jeff Gyurko]

Richard,

I would like to offer my condolences to the 2 that get to do it all if the IT Director has his way. We've got 6 individuals here that admin about 50-60 HP & Sun Unix servers.

The team is broken down into 2 junior level admins who deal with the mundane and 4 top notch admins who deal with the more complex issues that crop up daily. The 4 top notch are broken down and responsible for different business areas and the servers that they may use. This way the business gets comfortable with the same person and that same person gets extremely knowledgeable to the ins and outs of the specific business needs.

Although we can help in other business functions when the time calls for it, it's the main persons responsibility for that business. You don't have to break it down this way, but we feel it helps to have an individual to go to instead of anyone in the team.

Since we have more HP's than Sun's one of us 4 is more knowledgeable in the Sun arena than the HP's.

To break them down by hardware/software will be difficult since you might not know if a particular problem is hardware related or software that needs a patch. Eventually both people will be involved in a problem anyway so if you break it down in this manner, you'll be wasting time waiting for the other person to free up to look at a problem.

Also, when major releases of the OS come out, the software person will never have a break and the hardware person will just be sitting around. When the software guy leaves, the hardware guy will bear the burden of software too and not know much about it and vice versa. I would pass on that type of environment if I were "looking".

My $.02

Jeff

[A. Clay Stephenson]

Hi Richard,

With your number of boxes at the very least you need at least one more person. Dividing tasks along hardware and software especially in the UNIX world doesn't work very well as a general rule. It simply leads to a lot of finger-pointing. If you are divided along those lines, who does the maintenance contracts (which include both a software component and a hardware component)? The other thing to be aware of is this thing called vacation, leave, illness. If you are divided as you suggest then when the 'hardware' guy is out you are in trouble and vice versa.

More than most environments, you really have to have a good working knowledge of both hardware and software. It's very difficult to know what's an OS problem, what's a network problem, or what's a true hardware problem without quite a good bit of expertise in all of these areas. In some rare caes, the answer is actually all three at once.

If it were me, I would make sure that everyone is cross-trained. I do for all my admins. For what it is worth, I also find that the very best admins can at least spell C and are familiar with system calls. That knowledge really comes in handy when you are trying to tell a bunch of developers what is wrong with their silly piece of code and why it is a dog or at least why they are sometimes getting this silly errno value.

For what it's worth, Clay.

[Santosh Nair_1]

For what it's worth, I think its much better to have two or three well rounded ppl that can handle hardware, OS and applications related issues than to have 5 ppl with spilt responsibilities.
For one thing, ppl like a challenge and doing just one of the three aspects of the job would really get boring really fast. Not to mention the fact that one can back up the other in times of need.
Also different ppl can have different perspectives on problems. This can come in very handy. Many a times I've stared at a problem for hours only to have someone come along and point out the answer in a matter of seconds by looking at the problem from a different persective.

-Santosh

[Animesh Chakraborty]

Hi Richard,
Suppose you have got 50 boxes and 5 sysadmin. then divide 10 boxes each.Most critical production boxes to senior sysadmins and developements boxes to the newly joined one.
They will be primary responsible for their boxes for any kind of issue and also assign a secondary responsible person for each box in case of primary is not available.
But I think most important thing is to have team effort because one person may not know every thing.Coordenation among team playes makes lot of difference.
Thanks
Animesh

[Michael Tully]

Hi Richard,

In our company we have 4 SA's looking after
various types of systems, HP, AIX, NCR and Linix.
Our team comprises a variety of experience,
so when there is a problem on one type of
server there is always somebody to assist
with the problem if needed. I feel that you need at least one more person to assist, I also think that splitting such a small group of people would be counter-productive. What if one got run over by a bus, does that mean the other has to do twice as much. A number quoted to me some time ago was that an ideal ratio is 10 systems per SA. This number also depends if you may need to look after applications as well. Just remember IT directors don't have to sit there and do the work, they direct traffic.

My $0.03

-Michael

[John Waller]

Richard,

As Tim pointed out , once a UNIX box is setup correctly it runs on its own. (unlike a popular GUI, PC based O/S which people pay a fortune for).

One main points to consider when setting up a system admin team is where do the responsibilities lie. You have not given any details about the rest of your IT setup, but do you have a UNIX Tech Support team or will that be your responsibility. Most of my time is spent supporting users, either with scripts , changing permissions , checking print queues, etc.

The only thing I really do on a day to day basis on my UNIX systems is to check to ensure the backups have been completed, that no filesystems are running low on space and their are no unsavoury messages in the syslog files.

I have to agree with other reponses, I feel it is a bad idea having a seperate hardware/software team. You really need people adapt in both areas. To be honest most hardware is just connecting a couple of cables and switching on , the hardwork is done at a software level with configuring the device and testing it works correctly.
It may be an idea to split your duties between HP and SCO functions as sometimes its difficult switching between the 2 O/S's as their are annoying subtle differences between the 2, but all people should be confident in both.

You also mentioned about running cables, does that mean you don't have a seperate Network Team ?? If you don't the you could find that managing the network could be a job on its own for 1 or 2 people depending on the size of the company.

Without and Tech Support and Network issues, you could probably beable to to provide a 5 day x 10 hour basic day to day system admin cover on all your system with 2 people. Any other duties or hours then add 1 or 2 people for each extra function or work period. Plus make sure you are not expected to upgrade / patch all you system in one go. I feel it is not advisable to try to upgrade more than one system at a time as you get lost with what stage you are at on which system.

Well thats my 2p's worth (Yes I work with Pound Sterling)