topic Re: how many sys admins does it take .... ? in Operating System - HP-UX

how many sys admins does it take .... ?

John Kittel — Thu, 11 Sep 2003 22:30:44 GMT

I'm a forums novice, - I hope I'm not violating forums etiquette here by asking a (possibly) frivolous question.

In the collective experience of forums participants, how many system administrators per system and per company is typical?

For example, we have one sys admin (me), for approx 60 VMS, 10 linux, 5 Digital Unix, 5 HP-UX, and a dozen or so other various legacy systems. There are a few individuals here who are able to pinch hit in various areas when I'm absent, but they normally have other job responsibilities and so don't contribute to sys admin on a daily basis. ( I'm feeling overworked).

I don't know how I should assign points to responses to this question. Suggestions?

- John

Re: how many sys admins does it take .... ?

Patrick Wallek — Thu, 11 Sep 2003 22:38:00 GMT

We have approximately 40 HP-UX systems and there are 2 of us unix admins.

We have some other Windows / Netware systems and our Network equipment and there is a team of 3 other folks that manage those.

It just depends on your environment and how much of a glutton for punishment you are. ;)

Re: how many sys admins does it take .... ?

A. Clay Stephenson — Thu, 11 Sep 2003 22:41:10 GMT

The answer is, of course, "it depends". I can't really give a correct "score" for 60 VMS boxes but if 60 VMS boxes ~ 30 UNIX boxes then you probably have your hands full. It's made more difficult in that you are supporting several flavors of UNIX. Your job could (and almost certainly should) be shared between two competant admins.

Absolutely essential to your job is having an automated, completely hand-off backup system (e.g. OmniBack/Data Protector with a Library or two). Another "must have" is a product like VantagePoint/Operation (VP/O) so that you see problems and can often automatically deal with them as (or even before) they happen.

With that many boxes, I would certainly have an automated way of dealing with logins, home directories, services, hostnames, etc. -- read into that NIS/NIS+/LDAP.

Re: how many sys admins does it take .... ?

Sridhar Bhaskarla — Thu, 11 Sep 2003 22:48:13 GMT

John

I agree with it depends opinion. In some sites, you will not be allowed to maintenance during the office hours. So, even to install a commands patch, you will need to wait until a maintenance window in the middle of the night. But you are expected to be there during the day also because that's when the production happens. So, even if you have one such system, you will need to have two administrators.

If you feel you are too overworked, then you may consider buying softwares like Tivoli, Patrol, IT/A, VPO etc., that can allow you to do centralized administration.

For me, I will be happy to handle any number of systems provided if I am allowed to do maintenance during the business hours. But that's not reality.

-Sri

Re: how many sys admins does it take .... ?

Carla Breuer — Thu, 11 Sep 2003 23:20:53 GMT

Hi John,
I am the only sys admin / programmer / analyst at my account. There is nobody when I go on vacation, call in sick or anything else that might come up to fill in the gaps. They called me to work from my Grandma's funeral. What's wrong with this picture? I needed that little bit of stress relief.

Thanx,

Carla

PS: Nothing like blowing off a little steam to relieve a little stress.

Re: how many sys admins does it take .... ?

Con O'Kelly — Thu, 11 Sep 2003 23:24:53 GMT

Hi John

As everyone has said there is no correct answer to the question and it will very much depend on how the systems have been setup, the user generated workload such as requests from DBA's and the type of environment (mission critical 24x&?). Certainly in that I'd look at having at least 2 Sys Admins.

In your situation I'd put as much effort as possible in to automating as many sysdmin tasks as is possible (eg through scripts), so that you are automatically notified of filesystems nearing capacity, performance issues, key processes not running, logfile monitoring etc.

I'd also try & train other people to perform basic sysadmin duties such as adding new users, reseting passwords, dns updates, printer setup etc. For example you could then set a procedure in place (with management approval) that all User Admin etc is handled by another person in which they follow a predefined set of steps.

Cheers
Con

Re: how many sys admins does it take .... ?

Tim Adamson_1 — Thu, 11 Sep 2003 23:33:29 GMT

I agree with the other replies in that "it depends".

I have found that typically 1 sys admin should be able to effectively administer somewhere between 10 and 20 Unix servers.

However, if major projects are underway and/or the site actively upgrades software, applications, Operating Systems, etc then I think it can be too much for one person.

It is also essential to ensure you have effective tools and/or scripts to assist with common problems.

Tim.

Re: how many sys admins does it take .... ?

Steven E. Protter — Fri, 12 Sep 2003 00:21:03 GMT

I think a little too much is expected of you. If the legacy systems are stable and require little day to dday care, you're probably okay, but let me tell you about my schedule for HP-UX, which I consider the minimum for staying current.

BTW, I manage five HP-UX systems(1 educational at home), two Linux production systems for my webhosting business. I also have a few private clients with a few Windows Servers and about 20 workstations.

Daily tasks:
make sure all production oracle databases are up and healthy with regards to disk space.
make sure legacy databases and apps are up and working.
Check the email logs for critical system alerts that go out by email
scan the log files for possible security violations and stuff
Deal with telnet users that Operations can't handle(overflow).
Project work(currently planning an oracle upgrade).
Manage quality control on oracle applications improvement project(admit thats unique to here)

Weekly tasks:
Download and prepare installation of security patches required by security_patch_check
Check software.hp.com and download and prepare for installation.
Test patches and applications previously installed during prior maintenance period.
Validate Ignite backups and prepare for tests.

Thats just what comes to mind this week. There are dozens of other things I do without thinking.

If you're going to stay on top of things and keep your systems current, its tough to do with that many OS's and boxes.

What happens if there is a major problem like B
laster getting inside your shop?(Thank you Cisco consultant). That could require a log check and network configuration changes on all 100 boxes if you're unlucky.

Obviously managment is stretching you too thin to possbily maintain quality control. I hope they at least pay you well.

SEP

Re: how many sys admins does it take .... ?

Glen Trevino — Fri, 12 Sep 2003 01:42:23 GMT

I work for a large nationally recognized hospital and our "UNIX Team" consists of 4 admins (3 experienced 1 green) which support:

20 Prod/25 Test HP-UX servers
1 Prod/1 Test Tru-64 servers
1 Prod/1 Test AIX servers
2 Prod/2 Test VMS Servers (Just Backups and Disaster Recovery, application vendors handle the rest on the VMS boxen)

In addition, we manage and code the HL/7 interfaces (about 40), manage and code the enterprise Single Sign-On application, admin our external DNS and provide Unix support for all application upgrades (PeopleSoft, Lawson, several HBOC apps, GE Medical apps).

We are never NOT busy and there is never enough time in the day to get done what must. If we weren't salary we'd all be very wealthy from overtime. But it looks like you have a lot on your plate for one admin. 60 VMS servers!!! That's a situation I'm going to have nightmares about tonight!

Re: how many sys admins does it take .... ?

aparna challagulla — Fri, 12 Sep 2003 03:11:13 GMT

hi steven,

when u say that u check the health of oracle database by checking the disk space. can u pl tell me exacltly how do u do it. we have oracle 9i RAC on Hp-ux 11i MC serviceguard cluster.

regards,
aparna

Re: how many sys admins does it take .... ?

Jeff Schussele — Fri, 12 Sep 2003 03:21:38 GMT

...To change a light bulb....
Just one, but it takes three days of fighting Change Control to do it & then Mgmnt questions whether the ticket was in bypass at the time.

Tongue in cheek,
Jeff (No I've *never* been theeeerrrreee)

Sorry, It just had to come out......

Re: how many sys admins does it take .... ?

Tom Geudens — Fri, 12 Sep 2003 04:37:11 GMT

Hi John,
Now why would this be a "frivolous" question ... ;-) ?

Anyhow, we are 8 SysAdmins strong for the following :
- 300 Retail (remote) HP-UX 2 server clusters
Upgrades on these servers are completely automated (they are all exactly the same).
- 25 Central (local) HP-UX servers. Upgrades on these servers are manually (working on automating this as well).
- 10 Retail (remot) Linux servers
Upgrades on these servers are completely automated (they are all exactly the same).
- 10 Central (local) Linux servers. Upgrades on these servers are manually (working on automating this as well).

Next to our group we have an 3 men strong A.I. group (Application Infrastructure) that is responsible for the softwares and an 6 men strong database group that is responsible for the databases.

I would say you need at least another SysAdmin with the same experience as yourself and one trainee SysAdmin as well.

Regards,
Tom Geudens

Re: how many sys admins does it take .... ?

Zigor Buruaga — Fri, 12 Sep 2003 06:07:46 GMT

Hi,
We are 2 sysadmins for the following:

- 10 HP-UX Servers.
- 60 W2K Servers.
- 10 Legacy Systems.
- SYBASE and M$ SQLServer DBA's.
- 10 routers.
- Several applications ( some of them only maintenance, others build and maintenance ).
- TSM installation and maintenance for several W2K Servers.
- PLC programming projects.
- Operations support, configuring and building monitoring tools, etc.

In short, all sort of things are/will be under our responsability.
It's more funny, and you learn a lot of things ( that's the reason why you receives almost all the job! ), but is also more stressing, specially when one of us is on vacation ( I'm on vacation now ;-) ).
About your question, I think you need other sysadmin if your systems are critical. And, in the other hand, if I were your manager, I will put other sysadmin, it's not prudent that only one person support all that job, is not good for the worker and is not prudent for the company. Only my opinion.
By the way, I think you are right ... you are overworked.

Kind regards,
Zigor

Re: how many sys admins does it take .... ?

H.Merijn Brand (procura — Fri, 12 Sep 2003 07:22:58 GMT

I'm doing sysadmin for

1 x HP-UX 11.11
3 x HP-UX 11.00
1 x HP-UX 10.20
1 x AIX 4.3.3
2 x SuSE 8.2
1 x Win2k (my own PC)

many databases, pretty simple network.

Dial-in dial-out is managed by someone else
All other PC's and the only Mac here managed by yet another

And I'm with most of the above: it all depends. Mainly also on the user demands. If you have stupid users, who what you to restore `lost' data day after day, you will obviously have no time left for maintainance, analysis, precautions, and updates. Let alone for the time spend on the phone explaining why something does not work, or why things fail.

Another thing is the management. If they allow you to tell the users: "System down between 13:00 and 14:00 for maintainance", and you don't pass these borders, you're off much easier than management that tells you to do maintainance in off-hours.

Enjoy, have FUN! H.Merijn

Re: how many sys admins does it take .... ?

Keith Bevan_1 — Fri, 12 Sep 2003 13:48:24 GMT

John,

Same as previous responses.

It really depends on the user base, number of systems, company IT Infastructure, configuration, staff skills, training, documentaion and procedures etc etc.

We are lucky and have one for one.
One server one admin ratio !

10 servers 10 admin staff and plenty of cross over and responsibility.

Keith

Re: how many sys admins does it take .... ?

Geoff Wild — Fri, 12 Sep 2003 14:04:02 GMT

It depends...how much change in your environment happens, what kind of Applications, size, etc...

We have:

30 HP-UX, 8 in 4 x MC/SG clusters.
11 AIX
13 Solaris
4 EMC DMX Frames

Supported by:
8 Sys Admins (3 Seniors, 1 SAN, 3 Intermediate, 1 Junior) in a 7x24 environment.

Applications:
Point of sale
SAP (one of if not the largest in Canada)
Approx 700 printers, 5000 users
Retail System
Points System
OVO
Netbackup
DNS
NFS
Samba
etc...etc...etc...

Jeff - 3 day Change Control? lucky - we have a 5 business day change control....

Rgds...Geoff

Re: how many sys admins does it take .... ?

Fred Martin_1 — Fri, 12 Sep 2003 14:16:15 GMT

We have 1 HP-UX server, 6 NT servers, 140 PC desktops, about 20 text-only terminal users. There are 2 of us here....

Yeah we're busy but it's the desktops that take all our time.

Re: how many sys admins does it take .... ?

doug mielke — Fri, 12 Sep 2003 14:19:45 GMT

Our shop is undermanned as well, and 24 x 7 'follow the Sun' but it's tolorable when we're not making any changes. ( 1 sysadm, 12 Unix and 20 or so central Wintel boxes, hundreds more Wintels scattered about the globe, with some local folk to help with those)

Dept Rules:
Automate everything
Change nothing
Redundantly back up your redundant backups.

Re: how many sys admins does it take .... ?

GK_5 — Fri, 12 Sep 2003 14:42:25 GMT

"It depends" on so many factors.
1. How static/dynamic the enviorment is.
2. How conservative the site is about patch management.
3. How consious the site is about security issues
4. what is the policy about audting, upgrade etc.
5. How critical the systems/application are

Sorry dude, no confirm answer.

Re: how many sys admins does it take .... ?

Geoff Wild — Fri, 12 Sep 2003 14:48:56 GMT

PS: here's some info on "points":

http://us-support2.external.hp.com/estaff/bin/doc.pl/forward/screen=estaffAssistance?Page=file0002#forpoints

Rgds...Geoff