- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Re: Degraded backup and Oracle dump performance af...
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-30-2010 05:54 AM
тАО09-30-2010 05:54 AM
Degraded backup and Oracle dump performance after 11.23 to 11.31 upgrade
I am seeking for a solution of this i/o performance problem after upgrading from 11.23 to 11.31.
System rx6600 with EVA (HSV210) storage.
Now running HP-UX v. 11.31 Sep09 Dist.
Before running HP-UX v. 11.23 Sep06 Dist.
The nightly RMAN backup job og aprox. 2 Tbyte into separate disk are took before upgrade 5 hours.
Now the same job takes 8 hours.
I have testet the /exp with dd on another machine (same setup, just movig the vg04 over) )and get the same dd result.
Kernel params are set after Oracle recommandations.
I am looking for inputs from other sysadms with sort of problem.
Right now I am looking into vxtunefs.
Regards MArtin R├╕nde Andersen Miracle A/S miracleas.dk
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-30-2010 06:04 AM
тАО09-30-2010 06:04 AM
Re: Degraded backup and Oracle dump performance after 11.23 to 11.31 upgrade
did you have changed the Mount Options from the volumes (cache or without)?
Did you have change any Oracle Paramter (SGA etc..)?
Could you monitor the Disk queue during the Backup (with glance)?
by,
butti
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-30-2010 07:03 AM
тАО09-30-2010 07:03 AM
Re: Degraded backup and Oracle dump performance after 11.23 to 11.31 upgrade
Also did oracle get upgraded as well or stayed the same?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-30-2010 12:46 PM
тАО09-30-2010 12:46 PM
Re: Degraded backup and Oracle dump performance after 11.23 to 11.31 upgrade
What else, besides the OS has changed? SGA? Kernel Params?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО10-01-2010 03:13 AM
тАО10-01-2010 03:13 AM
Re: Degraded backup and Oracle dump performance after 11.23 to 11.31 upgrade
rx6600 ,Memory 32G , Cpu 2 x dual core Itanium
The test config (a similar machine), has been running clones of the database for 16 months.
The production database setup is the same:
- no changes in SGA, oracle
- I have not used any special mount options.
(what do you mean ?) I tested before the upgrade, moving the mount point between the prod (11.23) and test (11.31) and all application functionality was aproved.
- After going into production, we experienced that nfile=30000 was throttling the system and we changed it to 0.
- I am at the moment investigating disk que figures from the glance collections I have (1 minut interval) from before and after the upgrade.
- I tried (with no success to change vxtunefs params: for the /exp area. (now changed back after test overnight.
read_ahead=2 (now back to 1)
max_buf_data_size=64k (now back to 8k)
scsimgr param for the disk182 (/exp):
leg_mpath_enable=false (now back to true)
-I have attached the current varux01 (prod) kctune output, in which you also will see the changes I am planning to get changed at next reboot.
Thanks for the comments, sorry for the late answer back,I was hung up at another customer.
Regards, Martin Roende Andersen
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО10-01-2010 04:22 AM
тАО10-01-2010 04:22 AM
Re: Degraded backup and Oracle dump performance after 11.23 to 11.31 upgrade
Things done, that takes longer:'
RMAN backup BEFORE:
runs from 18:40 - 23:59
RMAN backup AFTER:
runs from 18:40 - 01:30
Changed in schedule
runs from 17:00 - 00:00 (aprox)
----
After RMAN backup to /exp, the /exp mount point is backup up.
BEFORE:
DP backup to fiber connected Tape:
Runtime 5:09:21
Backup speed 106.181,44 KB/s
Endtime 07:16:37
AFTER:
Runtime 8:05:55
Backup speed 66.29 MB/s
Endtime 10:05:55
Thats how the customer sees the problem.
Regards Martin Roende Andersen
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО10-01-2010 04:44 AM
тАО10-01-2010 04:44 AM
Re: Degraded backup and Oracle dump performance after 11.23 to 11.31 upgrade
Is the 11.31 production rx6600 a different server than the rx6600 11.23 production?
Also unless you have a typo, the DP times are the same (actually the AFTER is shorter but the speed is different. How is that happening?
The /exp is the common factor for both the RMAN and the DP backups but is that really the issue? Have you done any disk tests on /exp?
Also whether the hardware changed or not, I would look into the fiber interfaces (and the fiber in general). This is common to both the /exp RMAN writes and DP reads as well as the tape writes.
Have you noticed anything in the glance metrics? It might be better to observe the glance during the RMAN and DP runs so that you will see any alerts.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО10-01-2010 05:07 AM
тАО10-01-2010 05:07 AM
Re: Degraded backup and Oracle dump performance after 11.23 to 11.31 upgrade
- How similar? Completely identical (same model, same number and type of CPU, same memory, same FC cards etc...)
These jobs look like big sequential IO jobs - in my experience nothing much you might do with vxtunefs will change that - it would be helpful to see the mount options on the 2 servers:
mount -p
HTH
Duncan
I am an HPE Employee
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО10-01-2010 05:22 AM
тАО10-01-2010 05:22 AM
Re: Degraded backup and Oracle dump performance after 11.23 to 11.31 upgrade
It's possible all the ownership of LUNs has moved to just one controller (whcih would tally with about half the sequential IO performance you are seeing)
on 11.11/11.23 the EVA Vdisk controller preference should be left at ├в No Preference├в and the LVM Primary/Alternate paths should be set carefully to be balanced evenly across the two controllers and ports (because the UX IO will always be down one path only). (I├в m assuming Securepath was not used). This way UX determines the balance of IO and handles failover/failback, and the EVA should match Vdisk ownership to the prevailing IO pattern.
in contrast, with 11iv3, the EVA Vdisk preferences should be set deliberately to a 50/50 mix of ├в Controller A failover/failback├в and ├в Controller B failover/failback├в and Multipathing should be set to RR with ALUA (the defaults). In this case the UX server respects the EVA preferences/balance.
SO you need to look at the EVA CommandView config for these LUNs and then compare to what you see from evainfo and "scsimgr lun_map" on the 11.31 host, to be sure that all your IO is using both EVA controllers (any given LUN will just use 1 controller, but you should have an even number of LUNs spread across both controllers.)
HTH
Duncan
I am an HPE Employee
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО10-01-2010 06:13 AM
тАО10-01-2010 06:13 AM
Re: Degraded backup and Oracle dump performance after 11.23 to 11.31 upgrade
They are exactly the equal machines ..
Your point about the EVA has not been into consideration, and I will talk to the storage manager..
With /exp on varux01 machine:
/dev/vg04/exp1vol /exp vxfs ioerror=mwdisable,largefiles,delaylog,dev=40040001 0 0
From fstab:
varux01 # grep /exp /etc/fstab
/dev/vg04/exp1vol /exp vxfs rw,suid,largefiles,delaylog,datainlog 0 2
See the atttached file for vginfo, dsf, scsimgr info ..