daily tasks for a system admin ..

[someone_4]

Before you guys hang me ..!! I know this has been asked several times and that there are several posts and links about this. But with the new search .. I cant find anything =(
I also know I have asked this question before I found the link. But most of the answers are just links to links to links. So I am asking for some help on this. I need a list or routine task that a system admin does on a daily and weekly basis. As many of you know I have been working really hard to get my title of "System Admin". I am really the only person that has been taking care of my servers for the last year (sad but true).There has been great progress made. I have done allot of things that have made our 3rd party apps run 100% better then they have ever ran from day 1. Well Wednesday I decided that I was not making any headway in my carrer and Thursday I had a meeting with Management and told them that my last day would be Dec.28th. Why so long you may ask ? Well first I dont have a job and this would give me enough time to find something. 2nd I love my company too much to just up and leave. I wanted to make sure there wsa someone who could take my place as " The system admin without the title". Well the good thing is that they dont want me to leave. I explained my sitution and how I feel about everything. I made it clear I was not happy with my support/call center role. And that I am a System Admin. Well the end result was that I no longer have to answer customer calls !! WHOO HOO!! .. And I dont have to work customer trouble tickets.YEAH! I was told I can focus on System Admin task. They also asked me to get a list for them of duties and responsiblites that I would take on as The System Admin. We have about 50 HPUX server about 40 sco servers. With all this .. we have never had a true System Admin! Funny story .. I have been working with glance installing the trail versions.After digging around I found full licences for glance , online JFS and allot of other stuff that has been unopened!! so I am sure I can find enough to keep me busy on a daily basis if I only knew what to do. So that is why I turn to my fellow forum family.

Thanks in advance

Richard Leon

[John Bolene]

I normally do a daily bdf and see if any file systems need tending to. Sometimes it is files in /tmp or /var/tmp, sometimes print jobs in /var/adm/spool.

I check the backup job on Mondays after it ran on the weekend.

Sometimes syslog needs trimming.

Otherwise, I wait for users to call and say something is not working or they need their password reset as they have forgotten it.

[Bernie Vande Griend]

Standard answer is: it depends.

Here are a few references:
http://www.techsoup.org/articlepage.cfm?articleid=64&topicid=8
http://linux.oreillynet.com/pub/a/linux/2000/04/19/enterprise/CYA.html
Actually there is a lot of good descriptions out there, even amongst open job listings.

My answer is: sys admin should do everything necessary to keep servers and related equipment running and working properly according to its purpose and make sure end users have the knowledge they need to use it successfully.
Tasks: Systems security including server security, passwords, user accounts, virus software, security policies. Backups. Server installations including hardware install and OS install. Patch updates. OS Upgrades. Network adminstration. System Monitoring, watching log files for errors and correcting them, full filesystems, low memory, swap space, etc. Contact for vendor support. System performance tuning. End user training.
General troubleshooting for any system related issue. Application support for applications installed on server, or assisting application personnel as necessary. Same thing with databases.
Some of these tasks may be done by other people, but in general, these are all possibilities. It seems that since you've been doing sys admin stuff for sometime that you already know this though.
Are you looking to see if there is something you aren't doing that you should be?

[John Bolene]

Forgot to say that the above taske are done after the systems are optimized.

Use glance to check the resources on the systems, memory usage and cpu.

Also check to see that patch levels are close to current.

You may just be the performance guy along with sysadmin.

[Sachin Patel]

Hi Richard
Congratulation!!!!!!.

I have some scripts run on cron and sends email to us. checking bdf, top 5 process, defunct process, named, yp, other deamons, running jobs, old logins, some errors from syslog.log, zone transfers, yp maps, nfs mounts.

rest of the time I don't do anything.Unix works.

Sachin

[someone_4]

Yes I do most of thoose tasks. And yes I am looking for things that I might miss. I want to get good habbits and knock their socks off. One of the problems I have had was that I have had an identity crisis. One .. I was doing kernal configs,writting scripts,cron,basicly anything hpux related I would get a call about. Then on the other hand .. I would be working trouble tickets and answering customer calls about an 800 toll free number that was not working. Or one single pin # that was not working but when I tried it works. Pretty big gap there. Here is the website to my company so you can see what kind of bussiness we do.

if you wanna see flash
www.networkip.net
if not
http://www.networkip.net/home.html

[Eugen Cocalea]

Hi,

What others say:

http://www.usenix.org/sage/jobs/jobs-descriptions.html

http://wks.uts.ohio-state.edu/sysadm_course/html/sysadm-1.html#HEADING1-37

What I say:


A network (system) and a system administrator are like a family, where the sysadmin is the father and mother. Feed the kids. Take them to school. Bring them back. Help them do their homework (in the future, they will learn to do them by themselves, without too much interventions from you). Speak with the principal if they did something wrong. Apply some patches (well, ground them) if they went on a wrong way.


Maintain the network/system. Update/patch it. Clean around. Improve it. Get updated with the latest issues regarding the most aspects related to it (ie security, new OS releases).

ABUSE YOUR PARENTAL RIGHTS! You're the master, the kids... erm, computers, are your SLAVES.

(well, pet them from time to time)

AND (parents, ban this from your kids) (I couldn't help, this is fun - at least imho), check this:

http://unixsex.com/netadmin/

E.

[paul courry]

Richard,

I am attaching a copy of a paper I presented at HPWorld this year entitled "Damm Them Alligators" which includes 20 pages of code examples for automating proactive error detection and management. Perhaps it will be of use to you, most certainly it will give you and management something to discuss, particularly with regard to the necessity of support from upper management.

All the best!

[harry d brown jr]

These are rules, when I was a System Admin, that I lived by:

1) Automate everything
2) Offload everything
3) Pre-emptive is better than reactive
4) Work with Development or application providers to minimize the number of users that "need" (and "need" is questionable) access to a unix shell prompt
5) Write / buy / aquire monitoring tools - perl / glance / bigbrother / etc...
6) Document the crap out of everything. This allows you to monitor disk space, and when a user directory fills up, your support buddies can react to it, and with menus/scripts they can take care of the issue, making your life that much easier.
7) become the Capacity Planning engineer
8) stay on top of patches, and work with operations to plan maintenance downtime - this reduces unplanned downtime
9) Security - Security - Security - You must enforce security policies, and if you don't have any, work with the user community and create them!
10) And most importantly, get a few servers for your personal enjoyment. Play. get a PC and load linux. Have a blast!

live free or die

harr

[Sridhar Bhaskarla]

Richard,

I don't expect a system admin to daily logon to each system and run dmesg, check /var/adm/syslog, /var/mail/root etc., etc.,.

One should automate most of the tasks. There are some important areas that are to be proactively taken care.

1. Security
You need to buy/download softwares and keep them in place so that they send you mails regularly. There are good softwares available at porting site that can be downloaded.

2. Hardware and Software Issues
Software like IT/O, EMS, Predictive can be configured to alarm you whenever there are problems.

3. Performance
You can integrate default system utilities like sar/vmstat/iostat etc into nice graphing softwares like mrtg, rrd to get online graphs.
Also you can get some good softwares like sarcheck, BEST/1 etc and automate them to get good reports.

4. Capacity Planning

Based on 3, you can forcast the capacity of the systems to your management so that they will it be ready for future.

5. End user Calls

This is to be attented as and when required. This takes highest precedence over the above.

I would not suggest to you logon to each system everyday morning and run specific commands. It is highly unproductive. It may cost a bit to purchase some softwares but on a long run, it is going to save you a lot.

Best way is to configure the systems to send errors to a central place like e-mail so that the administrator can easily get notified and take action proactively.

-Sri

[Michael Tully]

Hi Richard,

Automate whatever you can, the outputs of
which are in e-mail form for yourself, or
if you have the facility to operators, snmp
traps e-mail.

Palm off anything you can to a Junior, so
that you can concentrate on the more
interesting things. This means that you learn
and then pass on the knowledge. This is best
part.... If you are interested in this stuff
it makes you have more of an incentive... even
getting up to go to the office.

Use 'sudo' for some mundane tasks, again for
operators, like user and printer management.

Hang in there Richard, your posts are always
interesting!

My 2 cents worth :^))

-Michael

[James R. Ferguson]

Hi Richard:

As you would expect, you have already received some excellent advice at this point in this thread.

Here are several links to some related, older threads. In chronological order:

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x6c91c3d7fb78d4118fef0090279cd0f9,00.html

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x7fcff841489fd4118fef0090279cd0f9,00.html

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0xe9e85f260cafd4118fef0090279cd0f9,00.html

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x7af3d211e18ad5118ff10090279cd0f9,00.html

Please note that most of the above threads contain links to other offerings. That, in part, was my objective. One thing leads to another. To be able to follow the embedded links, after the recent ITRC database conversions, change the substring:

"QuestionAnswer/1,1150,0x"

or:

"QuestionAnswer/1,11866,0x"

to:

"QuestionAnswer/1,,0x"

Using this amended URL will enable you to retrieve the reference.

Happy wandering!

...JRF...

[A. Clay Stephenson]

Hi Richard:

I think I'll go off on one of tangents again. The correct response to the daily tasks is nothing UNLESS your box tells you. The idea is to have a product like VantagePoint/Operations installed and it monitors your machines all around the world. It does take a lot of effort to get to that point and to cover all (or most) of the possible failure modes but when you do, you simply sit back and then no news is good news. You can even set it up to monitor NT and Novell servers and send the notifications to those admins. If a backup runs normally, you here nothing but if it fails you are notified. As an example, I was recently paged in the middle of a baseball game indicating that one of my HVAC units was indicating low pressure (due to a Freon leak); in that case, I did nothing because the 2nd unit in the data center could handle the full thermal load.

Having all of the routine stuff handled automatically, frees you up for one of the MOST important tasks of a good sys admin - planning what we will be doing one or two years from now and how do we get there.

Clay

[Roger Baptiste]

Hi Richard,

Automate the monitoring part as much as possible. That is the key!. The method
to do this will vary depending on the
size of the site, budget etc. For eg, in
an earlier site,(a small one) i had to
setup scripts to monitor important processes,
log files and send me a page in case of
an error, since i was the lone unix support for
the site. That was the most rudimentary
form of monitoring. Whereas in my current
site, we have a whole department taking
care of the ITO monitoring, 24 hour dedicated
on-call and other good stuff.

The reason why everybody is stressing on
the automate monitoring part is, it frees
up the headache of worrying about being
blindsided of being unaware of a latent problem.

Apart from system monitoring, backup is
another important issue. This again depends
on the site size, budget. The key though
is to have a solid backup policy which will
keep you safe in case of a disaster.
So, if your site does not have a clear backup
policy, this is the time to make it a
"project" ! ;-)

The next "task" is monitoring the
performance of a system. This also can
be automated by setting up perfview and
then taking it a step further by pulling
the perfview data to a graphical interface
which can be viewed on the web. That's
another "project"!

The tasks also vary depending on the customer base of the system and the business.
Customer meetings, project meetings , interacting with users, dbas, application dba's, network admins.. all of that takes
good chunk of time too.

Documentation is sometimes an unavoidable
necessity.

OFcourse, participating in forums such
as these is a VI "task" too!

Have fun!
R-

[Scott_14]

Hello There:

Well, I cant beleive you are doing system admin work, and the company doesnt undertand the postion, nor does it sound like they under stand the computer area.

The position duties all though have a lot of common things, can vary a bit from company to company. I suggest going to the book store, and just get a unix book, usually one geared twards admins, and usually the first couple pages, give you the System Administratiors job function, and then you can fill in more by all the support rolls and hats you have to wear.

This way, you get some good backing to back your self up on. The replies are long, cause most of us do a lot, sometimes so much we just cant list all of them.

good luck

scot

[Mark Vollmers]

Richard-

First off, congrats on the title (or condolences, depending on how you view it). I'd just like to throw out one more item, becuase everyone else has given so much. Don't forget, when you are bogged down with everything else, to GO TO THE FORUMS! After all, what kind of a SA would you be without throwing in your hat here on a regular basis! :) Good luck!

Mark