Linux X86-64 Machines (Nehalem, Dunnington, Shanghai, etc) versus UNIX Big Irons

[Alzhy]

I am seeing the march of BIG Linux X86-64 Machines as unstoppable versus their big UNIX machines.

Look at the recent TPC Figures... 2 Socket Nehalems -- breaking the 600K tpcc barrier. Theres even an 8-Socket (48-way) Dunnington Server breaching the 1 Million tpcc.

And it looks like recent SMP code in the latest kernels are becoming rock solid (with 256-way support) and oodles of memory.

The OS is nearly "free" -- on the average vendors charging ~ $x,xxx for the Advanced Server for 3 years at unilimited CPU socket compared to $xx,xxx for a UNIX machine.

I am currently testing an 8-Socket (32-way and 48-way) 512GB RAM and 2-Socket (8-way) 128GB RAM very large Linux Machines running RHEL 5.X and OEL. So far so good. All our StorageWorks SAN and Arrays fit perfectly, no driver issues, it just works. We can even adopt ou existing HP-UX/Solaris storage management standards and everything should be a go.

With virtualization choices on X86-64 richer than its UNIX counterparts -- the more compeling "Commodity Servers" plus Linux are becoming.

Your Thoughts?
How Large of a LINUX Environment are you currently running? (n-way and RAM?)
Are you running these environments at RHEL 5.X/OEL 4.X Releases

[Steven E. Protter · Accepted solution]

Shalom,

I'm not running anything other than a small office environment today.

Six months ago, I was working in a mixed environment with hundreds of x64 servers, and a goodly number of HP-UX systems, both PA-RISC and Itanium.

Here are my thoughts, a bit stream of consciousness.

These x64 boxes with smp kernels have become prime time. RHEL is still a product in need of some serious upgrade in Quality Assurance. That being said it is almost ready for prime time.

Cost of ownership I think is wrong.

I built a couple of labs for Oracle servers, high availability, Oracle RAC.

Cost of ownership on the RHEL/x64 side was much higher than expected. The servers were cheaper, but the OS support ran many hundreds per system, which we could not avoid because the customer would not accept CentOS versus Red hat.

Story of two labs.

RHEL 5.2 with MSA-1500 and dual fiber channel cards, switches and network. Cost around $40,000 to build, support costs were off a site license to RHEL but the cost per machine all told probably added up to around $1,000 per machine per year, plus my time, duplicating the Red Hat Satellite Network with shell scripts. 64 bit RHEL ES was used. Pair of Quad Core Processors in each.

Pair of rx2600 servers running HP-UX, oracle RAC, database hosted on NFS, no local storage around.

What surprised me is in spite of the HP-UX servers being under powered and out dated they were able to perform favorably versus the better equipped Linux lab.

I never locked down where the advantage was, HP-UX or the Itanium architecture.

The big disadvantage of the Itanium platform is that xen and will not work with the hardware. If you want VM you need to use HP-UX as the underlying OS and I don't know how that will go.

A very hard comparison, but don't count out big iron. In cost analysis after cost analysis, I found it was still competitive. The decision to go Linux had already been made, but for very good reasons the customer base, at least the part with enough influence to override our dictates, clung strongly to HP-UX. I think they had good reasons for that.

I don't give much credence to TPC. Its never even made me coffee.

SEP

[Court Campbell]

I really don't know where to begin. Anything can be done on Linux IMO. It really just depends on what you are trying to accomplish, and/or what applications you are running. I would think the really limiting factor would be the application. Most of my experience is with Oracle E-Business on UNIX. But this could easily be moved to Linux. We are currenlty looking at moving our apps tier to Linux. PA-RISC is nearly dead and the apps level we are at is not certified for Itanium. So Linux has become our choice, due to experience and cost.

[Alzhy]

SEP... Shalom.

May I know what Quad Core Intel Servers running Linux you compared your rx26xx (was it a Montvale 2 socket - 4-way system?) with? And local storage for the Linux versus NFS at that huh?

You should re-try pitting any IA64 or any POWER/SPARC system against the Nehalem Systems these days.

Specifically the Proliant DL 380 G6 -- which is a 2 CPU or socket, 8-way, 16 thread system. It broke through at more than 600K TPMC using the standard TPC suite. Other SAP standards would probably exhibit glaring and surprising results once they come out. When the Nehalem-EX 8-core comes out, it would mean the same 2 Socket FootPrint will be a 16-way, 32-thread beast.

I know SMP on Linux is quite young and scalability (and stability/reliability?) is still "suspect" but my tests with Nehalem systems (with the uber-fast QPI and DDR3 memory) is just astounding...

I do have my reservations too but in my case it is "client" driven too.

And the purpose of this thread is to really get opinions or even experiences.