- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Performance Issue on 11.31
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-22-2010 11:28 PM
тАО01-22-2010 11:28 PM
Performance Issue on 11.31
We have installed new Blade server BL860C with 11.31.Server is connected to EMC DMX4 storage and native multipathing is enabled.The problem I am facing is while I am doing a simple unix copy of 15g file ,it takes 6 mins and I am getting thruput of 45MB/S where as same file if I am copying in old server ie RP7420 hpux 11.11 it is only taking 3 to 4 min max and thruput is 65MB/s and there avserv is not crossing 4ms but new server is touching 18ms.I would like to know is there any special parameters need to set in OS level to get a high thru put.I am attaching here sar -H and sar -L output from new server while copy happens.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-23-2010 03:02 AM
тАО01-23-2010 03:02 AM
Re: Performance Issue on 11.31
So what sort of MPIO were you using on the 11.11 host? PowerPath? LVM PVlinks? nothing at all?
I ask because some disk arrays (and I can't comment on whether the DMX4 is one of those), don't do very well at handling sequential IO when those IOs arrive via differemt ports on the array. One big copy operation will generate just sequential IO.
If the 11.11 system is using PVlinks then actually all the IO will be going to just the one port on the DMX, whereas the default with the 11.31 MPIO will I expect be round-robin. If the DMX can't detect and optimise when sequential IO arrives down different ports, it could well end up being slower.
If this is the case you might want to change the load balancing policy - see this WP for how:
http://docs.hp.com/en/native-multi-pathing/native_multipathing_wp.pdf
I'd imagine that "preferred path" or "weighted round robin" might give better performance for sequential IO (if what you really want is better sequential IO performance - this might make random IO worse)
HTH
Duncan
I am an HPE Employee

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-25-2010 05:36 AM
тАО01-25-2010 05:36 AM
Re: Performance Issue on 11.31
Storage team done all EMC suggested bit for 11.31.
In 11.11 we are using PV links.I have tried to change the policy one by one but no net result .If I monitor my FC t-put I am getting 46MB/s for the first time copy but if I do the same copy second time it will shootup to 80MB/s.
I have noticed one more thing ,while I am using dd command ,server is giving good thru put of 70MB/s.I am not sure how it is.
dd if=/dev/rdsk/c14t9d7 of=/dev/rdsk/c16t9d6 bs=1024k count=500000.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-25-2010 07:14 AM
тАО01-25-2010 07:14 AM
Re: Performance Issue on 11.31
So probably a result of your filesystem buffer cache - maybe you have different settings on the filesystems you are using on 11.11 vs. 11.31 - compare using "mount -p" on both systems.
>> I have noticed one more thing ,while I am using dd command ,server is giving good thru put of 70MB/s.I am not sure how it is.
If you are using dd to the raw device you are again avoiding biffer cache, so this backs up that the effect you are seeing may not be disk related. What performance do you get if you run the raw dd on the 11.11 system?
HTH
Duncan
I am an HPE Employee

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-25-2010 07:21 AM
тАО01-25-2010 07:21 AM
Re: Performance Issue on 11.31
Also be awrae that unless you have set the load balancing to something different, then using the legacy device files will still cause round robin'ing between all paths to the disk.
You should figure out which agile DSF this legacy DSF is assocaited using "ioscan -m dsf", then post the output of:
scsimgr get_info -D /dev/rdisk/diskNN
replacing diskNN with the agile DSF you indentifed.
HTH
Duncan
I am an HPE Employee

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-26-2010 02:51 AM
тАО01-26-2010 02:51 AM
Re: Performance Issue on 11.31
I am attaching scsimgr info here.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-26-2010 03:07 AM
тАО01-26-2010 03:07 AM
Re: Performance Issue on 11.31
You're comparing a low end introductory BL860C against a RP7420 and your pissed because the blade isn't as fast as the high end super computer.
The RP7420 is going to out do the blade 4 to one in billions of processor operations per second. See below.
You gave up the horse power when you went low end econo class blade.
Computer
(Full Precision) HP 9000 rp7420-16 (1000MHz PA-8800)
/ HP Integrity BL860c (1.6GHz/18MB Dual-Core Itanium 2)
Number
of Procs
or Cores 16 / 4
Rmax
GFlop/s 47.5 / 24.48
Nmax
Order 30600 / 34920
N1/2
Order 1020 / 560
RPeak
GFlop/s 64 / 25.6
http://www.netlib.org/benchmark/performance.pdf
Note: You look at your sar report and you have nothing approaching a disk bottleneck. Absolutely no wait time what so ever. It's not in you I/O.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-26-2010 04:50 AM
тАО01-26-2010 04:50 AM
Re: Performance Issue on 11.31
(The rp7420-16 is only midrange, with up to 16 cores.)
>The RP7420 is going to out do the blade 4 to one in billions of processor operations per second.
Provided you can keep all cores busy.
Otherwise provided you don't need all 128 vs 48 Gb, a single core on BL860c will beat the rp7420.
It seems like this test case should be I/O bound but sar(1m) doesn't show it.
What does glance show?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-26-2010 05:53 AM
тАО01-26-2010 05:53 AM
Re: Performance Issue on 11.31
RE: "...(The rp7420-16 is only midrange, with up to 16 cores.)..."
You mean the high end superdome that starts with 1 to 16 cores, or the mid range rx8420 thats 2 - 16 cores? Or the rx7420 that 2 to 8 cores?
Which one Dennis? Or you just trying to be arguementative?
"...Provided you can keep all cores busy.
Otherwise provided you don't need all 128 vs 48 Gb, a single core on BL860c will beat the rp7420...."
Read the test material Dennis.
"...It seems like this test case should be I/O bound but sar(1m) doesn't show it...."
Dennis, if you don't know what 100% busy, or 0.00% avwait time is then maybe you shouldn't be answering this question.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-26-2010 06:27 AM
тАО01-26-2010 06:27 AM
Re: Performance Issue on 11.31
I would like to add that it is very difficult to compare these two systems.
An rp7420 has PA-RISC architecture and as noted, is a certainly not a low end system.
The blade has more modern processors, but has a completely different processor and back plane technology. So many things are going to be different, I/O capacity for example, even though the blade has a more up to date processor.
The original performance comparison is not realistic either.
A performance problem is real when applications show slow response time. Usually the end user provides the start of investigation with a complaint the system is slow. Your original file copy data does not surprise me.
If there is a user complaint, then its worth looking into and you may find the blade is not adequate for the task you have assigned it.
SEP
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-26-2010 10:49 AM
тАО01-26-2010 10:49 AM
Re: Performance Issue on 11.31
The problem is reported by Application team.This application is intensively use sequential IO activity and file will be big like 10,25 or even up to 50gb files.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-26-2010 10:51 AM
тАО01-26-2010 10:51 AM
Re: Performance Issue on 11.31
The problem is reported by Application team.This application is intensively use sequential IO activity and file will be big like 10,25 or even up to 50gb files.
HP engineer suspect this could be because of file cache setting.I am going to change it to check.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-27-2010 03:22 AM
тАО01-27-2010 03:22 AM
Re: Performance Issue on 11.31
HP's marketing blurb says rp7420-16 is only midrange. It's not a super computer.
http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&objectID=c00337654
>Read the test material Dennis.
Yes for that case but if you read my reply, I changed things.
>if you don't know what 100% busy, or 0.00% avwait time
I was agreeing with your sar(1m) interpretation, even though it didn't make sense for "simple unix copy of 15g".
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-27-2010 06:43 AM
тАО01-27-2010 06:43 AM
Re: Performance Issue on 11.31
I think possibly our questions and your responses have been a little bit hit and miss on this thread so far. If you really want to solve this issue we probably have to tackle it from first principles and then understand everything that is differemt between the 2 systems.
Lets start with what actually runs slowly.. you say it is a "simple unix copy of a 15g file".
Can you actually share the command line that you have been using to run your tests on. I refuse to beleive a "simple unix copy" can be CPU bound, and as such I'm going to continue to maintain we have an IO related issue here even if we see no avwait on sar outputs.
So I'd like to see for both the rp7420 and the bl860c all of the following:
1. the command you run to do the copy (just a cp?)
2. if you have tusc installed on these systems it would be nice to see the length of time spent in read/write system calls... e.g. if your copy command was usually "cp /dir1/file1 /dir2/file1" then "tusc -Cs read, write cp /dir1/file1 /dir2/file1" would give that data. You should try and collect this data on both systems immediately after the filesystem is mounted, and then see what differemce you get when you repeat the operation with a warm cache. I'm interested in this beacuse from the sar outputs you posted, all the long service timnes appear to be associated with read operations rather than write operations.
3. The filesystem mount options for both the source and target filesystem in the copy - also if you have anything in the /etc/vx/tunefstab
4. The LV and VG configurations for the source and target filesystems (lvdisplay -v / vgdisplay -v etc.)
5. the amount of physical memory and the kernel parms for buffer cache (dbc_min_poct and dbc_max_pct on 11.11 and filecache_max and filecache_min on 11.31)
6. the SCSI queue depth on all LUNs involved in the IO operations (use "scsictl -m queue_depth
7. You mentioned you changed to preferred path for the 11.31 load balancing - can you show me how you did that (it's easy to get wrong...)
there's probably more, but that should get us going...
HTH
Duncan
I am an HPE Employee

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-27-2010 09:50 AM
тАО01-27-2010 09:50 AM
Re: Performance Issue on 11.31
"...application is intensively use sequential IO ..."
Please post 'sar -b' reports.
Question: Is this a database or a series of flat files being written?
You should also attach all the regular performance analysis commands like vmstat, sar -d,u, etc.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО05-07-2010 01:02 PM
тАО05-07-2010 01:02 PM
Re: Performance Issue on 11.31
Were you able to find an answer to this issue? We are noticing similar i/o performance issues with the BL860C running 11.31.
Thank you,
James
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-02-2014 11:03 AM
тАО06-02-2014 11:03 AM