Survey: Outsource UNIX System Administration Offshore

[Stuart Abramson_2]

This is not a technical question. But I think that this is a good forum for asking about this.

A major metal-mining and manufacturing company, $20 Billion gross sales, Fortune 100 company, right here in my home town, Pittsburgh, PA, announced this week that is in the process of outsourcing the following IT functions, to be performed remotely, probably to a company in India, yet to be determined:

HP UNIX System Administration
Oracle DBA
Windows NT Administration

About a year ago, they outsourced the Help Desk to a company in Atlanta, GA.

Some staff will be retained to interface directly between the company and the outsource partner, but most of 20 people (my estimate) will be released. (Of those 20, some 5-10 are contractors.)

This is my first first-hand experience with outsourcing HP Unix System Administrators, and, frankly, it scares me.

Is this happening throughout the US? Is it happening in UK? Is it going well? Is it failing for any reason?

I'm interested if others have first-hand experience with this....

Interestingly, EMC Storage Administration, is NOT being outsourced. It is not considered a good candidate for outsourcing (I'm not sure why).

Stuart

[Dave Hutton]

The company I was working before looked into outsourcing most of our IT. I think the unix group was included at the time (At least they didn't say we were excluded). From what I recall though, all the companies were in the US.

But they had used outsourcing to India for other things like customer service reps, and some programming dev. So I'm sure they wouldn't of been against it.

My wifes company just outsourced most of their dev programming to India.

I believe its becoming more and more common.


Dave

[Steven E. Protter]

My opinion:

Short the stock.

I have no first hand experience with this. A lot of what I do is independent and requires not input from management. It could be outsources.

As far as resource planning and half of what I do. Face to face communication is critical. I meet with users, create technical requirements, do lots of things that require up front administration.

Plus: Remember these boxes have certain functions like single user mode that should be performed from the console. Yes, web console works, but sometimes you gotta put your hand on the power switch.

Sometimes you have to eyeball the little green lights. How is someone is Asia going to do that, with a webcam?

I think the company doing this is unwise. I could give them compelling business reasons not to do it.

There is some dataprocesing fucntion like support and programming going overseas. Dell just brought business customer support back because business customers won't tolerate the poor phone connections one often gets to certain cities in Southwest Asia.

I've had support calls with Oracle messed up by this. I've had a 16 hour support center in Banglore India sit on my call while they were offline for 8 hours and refuse to transfer the call to the US without a call to a Senior manager.

It is being done, it is likely to fail.

SEP

[Stuart Abramson_2]

Dave:

I have heard of and read about development programming moving offshore - just never UNIX System Admin.

I had thought that UNIX System Admin was too "hands on" or "too close to the machine" to be moved offshore.

Apparently there is IT management in the US that considers it feasible.

[Stefan Farrelly]

It is indeed happening in the UK. Just met someone yesterday from DHL who said their entire HP-UX server suite in the UK (hundreds of HP's) is moving to Prague! All sys admins here lose their jobs, theyve trained up sys admins in Prague to run them. Good luck I say - but they said it will save them a ton of money.

[Dave Hutton]

Stuart I see what your saying. But it started with customer service reps, then programming next... Whatever can save management money (I doubt it does. But I'm sure it looks good on paper). I wouldn't be supprised if it went toward the more hands on types of jobs.

I know it's going to backfire if the continue down this path.

My 2 1/2 year old daughter can turn off a computer or tell me if my green power light is on. I'm guessing pay someone to sit in your building cheap wages to be there all the time and remotely administrate it.

Dave

[RAC_1]

I have a firsthand experience on this.

I am currently involed with remote administration for a client in US. Doing from INDIA. Now this is my first time working on a such project. We do have some problems in that. Few functions as SEP pointed still needs someone at on-site to do it. And there are otions for this also. But most of the things can be taken care off remotely. The one problem is that one needs to understand work culture of onsite staff, understand work environment and setup"

We had most the problems with that. Other than this it is still a WIN-WIN situation for both.

Also besides all this, today or tomorrow, companies will have to think beyond cost considerations, like employement problems etc.

[Bill Hassell]

It is possible to perform HP-UX admin remotely, even in a lights-out scenario. VERY few companies get it right the first time, so the first questions to ask are:

How will the console be accessed?
How will this (special, isolated, highly secure, never shared with application network) connection be kept secure and redundant?
How will repair and upgrades be handled?
How will backups be done (are there operators on site?)
How will a disaster be handled? How long before anything meaningful can be started for recovery?
Is this a 24x7 operation? If so, what extra hardware and people are being made available to handle hands-on tasks?

Most outsourcing defines the easy stuff...the computers are all running just fine, the network is accessible remotely and there are no hackers already at work inside the system(s). The hard part is all the hands-on requirements. Repairing a broken box is also easy but getting it back online can be a nightmare without knowledgeable people onsite.

[Pete Randall · Accepted solution]

RAC,

I hardly see how you can call this a win-win situation. Obviously it's a win for the company providing the outsourcing. It may or may not be a win for the company contracting the outsourcing. It certainly is not a win for the SysAdmin who gets displaced. At best you might be able to call it a win-draw-lose.


Pete

[ian Dennison]

Stefan

Yeah, a m8 of mine in DHL has wangled himself a transfer to Prague, plus extra benefits. But he is one of the lucky (very) few!

It sounds like a political decision, not a technical or quality-of-service decision. Ergo, foregone conclusion, denials all around and blame the techies when it goes belly-up.

Methinks someone has seen "cheap" and "plentiful" and confused it with "quality" and "workable solution".

Likely EMC saw things like this coming and buttoned down the support contract with rigid get-out clauses or customer deliverables that prevent the relocation of the SAN support (Have dealt with EMC before, they are most certainly not dumb!)

12-18 months later it will all be turned around once heads have rolled and senior management have shuffled around some!

Cheers, Ian

[RAC_1]

Pete,

It is a WIN-WIN situation for both. Cost cuttings for a company outsourcing it and bussiness for company getting it. As I said earlier, it is win-win situation from economic point of view.

And with most of the companies stressing cost considerations these days, this is bound to happen. But yes employement problems that are created will have to be addressed. This is going to be a very important consideration very soon. the scale of outsourcing Unix sysadmin is not very high yet.

[Patrick Wallek]

I have to agree with Pete here. I don't necessarily see outsourcing as win-win either. I really think outsourcing the HP-UX portion is a BAD idea.

The problem I have with outsourcing, here in the US anyway, is that it is taking jobs away from US folks. People here complain about lack of jobs, unemployment, etc., so what do companies do... They eliminate US jobs and send them elsewhere. It just does not make sense to me.

I think a lot of companies are governed too much by the bottom line and not enough by common sense.

I am *NOT* trying to criticize *ANY* of the employees of the offshore companies that are taking the work. Far from it. If they can get the work, great.

It is big corporate upper management that I think sometimes doesn't have a clue.

OK, I'll get down off the soapbox now........

[Jeff Schussele]

Hi Patrick,

They usually have a clue within their area of speciality.
It's when they venture outside of their area that mistakes are made.
We've all seen what happens when Marketing-types make technical decisions. Now we're seeing Technical-types make decisions that will affect their markets - i.e. who will their customers be when those customers will no longer be able to afford their products?

And even furthermore this will inevitably be a pendulum that will swing back to the US when the average salary in India/Russia/wherever rises - yet falls here. THEN the jobs will come back.
But it sure reminds of that dumb ole Irish Setter down the street that spent considerable portions of each day unsuccessfully chasing it's tail.

My 2 cents,
Jeff

[Chris Wilshaw]

One of the most emotive topics for a while, and regularly a subject raised in various computing industry magazines.

As far as outsourcing is concerned, I know that there are some companies (including banks) in the UK that are actually reversing earlier decisions on this, and bringing the call centres etc back "home". This is following complaints from customers about the service they've received.

Most traditional businesses look at the short term gains that can be achieved. Of course, if saving money causes you to potentially lose customers, it's a false economy. Family and friends are very quick to desert a business that they perceive of having performed an act of betrayal.

When looking at the sysadmin role, the obvious point to consider is the one most of us know only too well - the role is anything but simple. Yes, the OS may be "standard", and relatively easy to support, but what about the applications that run on it? If you know that you're at risk of being outsourced, can you honestly say that you'd document everything, or train people up to a sufficient level to support the system? I doubt it.

As far as being a win-win situation is concerned, it couldn't be farther from the truth. Yes, the company can save some money, but what they lose in terms of business knowledge, respect, loyalty, reputation, rapport with end users etc can more that outweigh that.

[Joe Short]

Outsourcing can't even be considered and economic win. As an area experiences more and more unemployment, the lack of income will trickle down to the entire local community. People stop spending on things, and only purchase the absolute necessities, this hurts all of the local businesses that would otherwise be selling non essentila items. In the end, everyone gets hurt, as the busines that saved by outsourcing finds a shrinking market for their goods and services, it is all part of a complete circle. If one end is opened, it all falls apart.

[Stuart Abramson_2]

Reply to Joe Short:

When I was young, the US had many manufacturing jobs that have COMPLETELY DISAPPEARED: textiles, shoes, cameras, televisions, radios. Those old manufacturing jobs NEVER CAME BACK. Some industries still employ Amercians in America, but less of them: automobiles, steel.

For manufacturing jobs, the cycle was not reversible... And now, we're the next target...

[Joe Short]

Stuart,

God I hope not. Although as a consultant, that may bode well for me. The big picture is that it hurts us all. Unfortunately, big business is in control of the economy, and it is killing the little guy. One day there will be one employer, and we'll all be at it's mercy. I hope I'm oil by then.

[Mark Greene_1]

Anyone who views system administration as a commodity job clearly doesn't understand what it is to admin a system.

System administration is more than just maintaining the hardware and OS. I feel obligated to maintain a relationship with the applications folks, with the DBAs, with the help desk and desktop support people. I have more to offer the organization with my experience and opinions, over and above my hands-on work with the systems. I can be so much more effective when I have a vested interest in the organization's bottom-line and the opportunity to be pro-active with changes to improve efficiency and reduce risk.

Not that this cannot be done remotely or by someone who's a contractor, but the realistic levels of motivation to do so are so much less, from my observations.

mark

[doug mielke]

Let's assume that outsourcing is the way of the future. We've seen it in hardware design and manuf, software development, and now it creeps into operations.
What can we, as admins, do to prepare? Whats the next step for an IT pro?

[Tim Sanko]

Seen it, been there, called back to work.

There is only one good reason that they want to outsource Unix administrative duties. It is green and folds. Sooner or later, the lack of onsite support causes grief.

The problems that come out of it are these.

1. Outsourced talent for us was less than stellar. To save money the resources used were less than outstanding.

2 The routine stuff (adding users, and killing jobs, storage allocation) needs to be handled in a prompt manner when your admin is even in different timezones, it causes conflict...

3. Down systems support is woefull.
When there is less than 100% successful electronic communications, it is a sad stressfull time.

Since many of the places they are trying to outsource to are 8-12 timezones away...

4. Security. When you are having DBAs/Admins try to do the job from the other side of the world there are less reasons not to do it here.

True total cost. The long distance charges, and difficulties end up being language, cooperation, schedules, etc...

I don't know that NT outsourcing is that bad, but I have seen a yo-yoing of In-out-In at many places. I believe that more IT is starting to return due to the lack of ability os a manager to stand there and yell at you. "Why isn't my machin working???"

I have seen payrolls Data base Administration returning as well as senior Unix administration positions.

For the major reason of Management sanity, if for no other Unix Admins/DBAs are returning to the U.S.A. An admin 12,000 miles doesn't care if the server has a memory problem quite like a guy on the scene...

Tim

[Pete Randall]

Doug,

Move to India?

:^)


Pete

[Rita C Workman]

Hi Stuart,

I think I commented on this in another emailer...but why not again:

Read a couple articles on this kind of event. Seems it is the going trend.

My feelings, they will do this and companies will experience the issues that not having a good UNIX Admin on staff is most definitely going to cause....and probably go back to having in-house - - - eventually.
Something similar to the Consolidate IS / Distribute IS cycle that companies seem to go through about every 7-10 years.

It is amazing how some numbers crunchers think that all a UNIX Admin does is load patches. The fact is a good UNIX Admin is a Systems Administrator....which means understanding Systems/Interfaces/Technologies, etc....dealing with what folks think they want in a system, and what they really want.......see my point! We've all been there. Do you really think that guy in India has a clue as to what your environment and business needs are....and how to get your systems & technology to meet those needs. Nope..but those pencil jockey's will justify their paycheck by making such suggestions.

The ill effect of all this.....qualified people won't be there, when you need them. In there place will be the lowly operator who will simply be there to put the CD in the drive or load fresh tapes in the library.

And why not the SAN person......let's face it, there are probably a few more UNIX Admin's sitting around then SAN Mgrs. When that scenario changes...they will try to remotely manage disk as well.

Rgrds,
Rita

....and as the daughter of a all too often out of work "steel worker father". We have to earn a living in this country too. Heads up to HP (and all the other companies with this line of thought)...Keep moving out the jobs and I hope you understand that eventually there will be nobody left here to buy the hardware. We'll all have to move where the work is...
How do you say "...get a clue" in Indian (i.e. not Native American)

[Jeff Schussele]

Hi Pete,

Well that would work if it weren't AGAINST THE LAW for a foreigner to get a work visa in India.
At least a US citizen - for sure.
Ain't that nice of them?!?!?!

Wonder what they'd say if we reciprocated?
Somehow I know.......

Rgds,
Jeff

[Tim Sanko]

One more thought.

How long til you explain to a EMC storage guy that he has to toggle the Octal flag for HP.

If an HP guy isn't on site, There may only be 1/2 the EMC disk accessed. (I saw that somewhere. Until the understanding was reached with the storage manager only 25% of the Symmetrix capacity was available, and less when BCVs were invloved. this took place when they had decided to augment the staff with a gray-haired admin who had been through some close combat...)

Could an off-site Admin have done it? I don't know. Would an off-site Admin have done it? Why should he, he doesn't work for the company?

Tim

[Rita C Workman]

Pete,

Are you serious???????

I say that with 6 Indian temps working in IS positions (programming & DBA) at this very moment....out of all the temps hired only 1...just one is not here on a Work Permit. Just a relocate from Connecticut.

Talk about rude.....maybe the term 'ugly American' needs to be changed to something else.
Not that I'm picking on folks from India...but looks like these folks here could be heading back home and finding work just as well !

Rgrds,
Rita