- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - OpenVMS
- >
- Re: Extremely slow backup to LTO Tape after replac...
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-08-2014 04:30 AM - edited 08-08-2014 04:31 AM
08-08-2014 04:30 AM - edited 08-08-2014 04:31 AM
Re: Extremely slow backup to LTO Tape after replacement of hot plug (shadowed) internal SCSI disk
"No, I don't think so: the MONITOR data of EMU1 confirms, that the majority of the reads are from $3$DKA300: (the LOCAL member) and only SOME IOs are from $4$DKB300: (the REMOTE member). And the local disk $3$DKA300: is actually the one you've replaced."
I stand corrected, you are right. It's just the master member of the set because it fell into that role when the $3$DKA300 disk died. My apologies.
Could something as mundane as a io autoconfigure help here?
Or, the solution is to replace all the other disks and get wonderful throughput back again... :-)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-08-2014 04:39 AM - edited 08-08-2014 04:41 AM
08-08-2014 04:39 AM - edited 08-08-2014 04:41 AM
Re: Extremely slow backup to LTO Tape after replacement of hot plug (shadowed) internal SCSI disk
Mark,
you can change the Read Cost of the shadowset members on the fly.
Maybe start to experiment on EMU2:
EMU2 $ SET DEV/READ=1 $3$DKA200:
EMU2 $ SET DEV/READ=500 $4$DKB200:
Then try the BACKUP/PHY DSA2: test on EMU2. Most of the reads should now be from the EMU1 member and should be 'slow'. If that works as expected, apply the same setting on EMU1 for DSA1: and DSA2: by reducing the Read Cost for the (remote) EMU2 members and increasing it for the (local) EMU1 members.
To reset the read cost values back to their defaults, use $ SET DEV/READ DSA2:
And no, I don't think a SYSMAN IO AUTO will help here. Probably only a reboot or even power-off/on of EMU1. But maybe can can work around this problem by modifying the Read Cost...
Volker.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-08-2014 04:52 AM
08-08-2014 04:52 AM
Re: Extremely slow backup to LTO Tape after replacement of hot plug (shadowed) internal SCSI disk
Volker,
I did as you requested:
SET DEV/READ=1 $3$DKA200:
SET DEV/READ=500 $4$DKB200:
and for good measure:
SET DEV/READ=1 $3$DKA100:
SET DEV/READ=500 $4$DKB100:
Results (as you expected):
OpenVMS Monitor Utility DISK I/O STATISTICS on node EMU2 8-AUG-2014 21:50:17.29 I/O Operation Rate CUR AVE MIN MAX $4$DKB0: (EMU2) ALPHASYS1 0.00 0.12 0.00 5.00 $4$DKB100: (EMU2) DATA1 0.00 0.00 0.00 0.00 $4$DKB200: (EMU2) DATA2 0.00 0.00 0.00 0.00 $4$DKB300: (EMU2) DATA3 0.33 1.73 0.00 7.00 $4$DKB400: (EMU2) ADMIN1 0.00 0.00 0.00 0.00 DSA1: DATA1 59.00 37.16 0.00 76.00 DSA2: DATA2 0.00 18.73 0.00 76.66 DSA3: DATA3 0.33 1.17 0.00 3.66 $3$DKA100: (EMU1) DATA1 R 59.00 37.15 0.00 76.00 $3$DKA200: (EMU1) DATA2 R 0.00 18.73 0.00 76.66 $3$DKA300: (EMU1) DATA3 R 0.33 0.98 0.00 2.33
Curiosity got the better of me:
SET DEV/READ=1 $3$DKA300:
SET DEV/READ=500 $4$DKB300:
Output:
OpenVMS Monitor Utility DISK I/O STATISTICS on node EMU2 8-AUG-2014 21:51:47.31 I/O Operation Rate CUR AVE MIN MAX $4$DKB0: (EMU2) ALPHASYS1 0.00 0.15 0.00 5.00 $4$DKB100: (EMU2) DATA1 0.00 0.00 0.00 0.00 $4$DKB200: (EMU2) DATA2 0.00 0.00 0.00 0.00 $4$DKB300: (EMU2) DATA3 32.33 5.32 0.00 34.66 $4$DKB400: (EMU2) ADMIN1 0.00 0.00 0.00 0.00 DSA1: DATA1 0.00 24.77 0.00 76.00 DSA2: DATA2 0.00 12.48 0.00 76.66 DSA3: DATA3 2340.66 289.70 0.00 2361.00 $3$DKA100: (EMU1) DATA1 R 0.00 24.77 0.00 76.00 $3$DKA200: (EMU1) DATA2 R 0.00 12.48 0.00 76.66 $3$DKA300: (EMU1) DATA3 R 2341.00 289.57 0.00 2361.00
It's like the new disk has stolen all the bandwidth... :-(
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-08-2014 04:57 AM
08-08-2014 04:57 AM
Re: Extremely slow backup to LTO Tape after replacement of hot plug (shadowed) internal SCSI disk
Mark,
It's like the new disk has stolen all the bandwidth... :-(
Or all the lower SCSI IDs have some problem ? How about the performance of $3$DKA400: ?
The experiments with changing Read Cost have shown, that this would be a valid, quick and safe workaround to decrease your backup runtime, until this problem can really be solved.
Volker.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-08-2014 05:14 AM - edited 08-08-2014 05:18 AM
08-08-2014 05:14 AM - edited 08-08-2014 05:18 AM
Re: Extremely slow backup to LTO Tape after replacement of hot plug (shadowed) internal SCSI disk
Volker,
"Or all the lower SCSI IDs have some problem ? How about the performance of $3$DKA400: ?"
Nope. I recall I tested it. It came up as 81. A tad better than 76 (I'm an optimist).
OpenVMS Monitor Utility DISK I/O STATISTICS on node EMU1 8-AUG-2014 19:52:06.38 I/O Operation Rate CUR AVE MIN MAX $3$DKA0: (EMU1) ALPHASYS 4.00 1.10 0.00 24.66 $3$DKA100: (EMU1) DATA1 0.00 36.45 0.00 76.00 $3$DKA200: (EMU1) DATA2 0.00 8.28 0.00 76.66 $3$DKA300: (EMU1) DATA3 0.66 551.49 0.00 2834.33 $3$DKA400: (EMU1) ADMIN 21.33 7.75 0.00 81.00 DSA1: DATA1 0.00 36.45 0.00 76.00 DSA2: DATA2 0.00 8.28 0.00 76.66 DSA3: DATA3 0.66 548.33 0.00 2834.33 $4$DKB100: (EMU2) DATA1 R 0.00 0.00 0.00 0.00 $4$DKB200: (EMU2) DATA2 R 0.00 0.00 0.00 0.00 $4$DKB300: (EMU2) DATA3 R 0.66 11.09 0.00 50.33
I really appreciate all your help, Volker. Thank you.
I will use your suggestion of setting the read cost appropriately to preference the remote disks. That is a great solution/workaround.
(If the moderators and yourself, Volker, don't mind, I will leave this as unsolved and await the resolution of the case from HP to hopefully resolve this conundrum).
Cheers,
Mark.
Just a note for anyone reading this, the command to reset the read cost is:
SET DEVICE/READ=1 DSA1:
You may specify any read cost because the O/S ignores it and resets the read costs for each device in the shadow set to its default.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-08-2014 05:23 AM
08-08-2014 05:23 AM
Re: Extremely slow backup to LTO Tape after replacement of hot plug (shadowed) internal SCSI disk
Mark,
now if ALL disks on that SCSI bus EXCEPT the newly replaced one perform badly after the replacement, you might also have to ask yourself: is the replacement disk $3$DKA300 itself causing the problems ? Is that disk of the SAME type as the previous one ?
Volker.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-08-2014 06:06 PM - edited 08-08-2014 06:12 PM
08-08-2014 06:06 PM - edited 08-08-2014 06:12 PM
Re: Extremely slow backup to LTO Tape after replacement of hot plug (shadowed) internal SCSI disk
Hi Volker,
"now if ALL disks on that SCSI bus EXCEPT the newly replaced one perform badly after the replacement, you might also have to ask yourself: is the replacement disk $3$DKA300 itself causing the problems ?"
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-09-2014 06:09 PM - edited 08-09-2014 06:13 PM
08-09-2014 06:09 PM - edited 08-09-2014 06:13 PM
Re: Extremely slow backup to LTO Tape after replacement of hot plug (shadowed) internal SCSI disk
>I attempt to attach the .csv from T4 to which this junk software says:
>"The contents of the attachment doesn't match its file type."
I also get that attachment error and Excel likes that file fine. Your .zip solution is a good workaround.
Forum problems should be reported here:
http://h30499.www3.hp.com/t5/Community-Feedback-Suggestions/bd-p/community-feedback-suggestions
>If ... Volker, don't mind, I will leave this as unsolved ...
You can still assign Kudos to each helpful answer by clicking on the Kudos star.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-12-2014 07:04 PM
08-12-2014 07:04 PM
Re: Extremely slow backup to LTO Tape after replacement of hot plug (shadowed) internal SCSI disk
What specific sort of disk is it? SHOW DEVICE /FULL $3$DKA300: will usually give some sort of an identity for the device, otherwise there are some SCSI-level tools to gather information.
With V7.3-2, you have access to dissimilar device shadowing (DDS) and dynamic volume expansion (DVE), two features that are worth enabling when you next have some scheduled downtime or can otherwise dismount the shadowsets briefly.
36 GB disks are ancient, too. I've been scrounging new-old-stock 146 GB SCSI disk drives for ~US$40 for a while now. Amazon has available HP 72 GB 15K SCSI disks in what is likely the appropriate "Universal" disk sled for that AlphaServer DS25 for ~US$15, with free shipping. Several different 146 GB 15K HP drives on offer for ~$37 to $40 over there, too.
I'd look to patch to current V7.3-2 (or to upgrade), enable DDS/DVE on the disks, and start a migration to those 72 GB or 146 GB drives, and then let HP know you've upgraded your storage for purposes of the support contract.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-14-2014 09:54 PM - edited 08-14-2014 09:54 PM
08-14-2014 09:54 PM - edited 08-14-2014 09:54 PM
SolutionThe situation was finally resolved.
I placed a support call and after some procrastination by the people at HP that do such diagnosis they came to the conclusion that the tape drive controller needs replacing. What?!?!?!
This was because they saw errors logged in ERRLOG.SYS at the time I hot-swapped the drive. This was DESPITE the fact I told them, and showed them, the evidence I assembled above indicating all BUT the new disk are performing badly.
A HP Engineer came to the site bearing a replacement SCSI controller for the Tape drive.
Before he replaced the card he said the hot-swap was undoubtedly the problem. Other systems with RAID controllers we have are more forgiving of hot-swaps. Even though the hardware supports hot-swap, it seems OpenVMS doesn't necessarily behave well afterwards (I am paraphrasing the engineer).
So, a reboot solved the problem.
Here is the final output from running BACKUP/PHY/NOCRC/GROUP=0 disk: NLA0:x.x/SAVE on all disks in the system after a system reboot:
OpenVMS Monitor Utility DISK I/O STATISTICS on node EMU1 14-AUG-2014 00:03:54.70 I/O Operation Rate CUR AVE MIN MAX $3$DKA0: (EMU1) ALPHASYS 2403.66 272.33 0.00 2404.00 $3$DKA100: (EMU1) DATA1 0.00 429.30 0.00 2519.33 $3$DKA200: (EMU1) DATA2 0.00 297.22 0.00 2526.33
$3$DKA300: (EMU1) DATA3 34.00 341.23 0.00 2674.33 $3$DKA400: (EMU1) ADMIN 0.00 428.48 0.00 3497.00 DSA1: DATA1 0.00 429.30 0.00 2519.33 DSA2: DATA2 0.00 297.22 0.00 2526.33 DSA3: DATA3 34.00 341.14 0.00 2674.33 $4$DKB100: (EMU2) DATA1 R 0.00 0.15 0.00 12.00 $4$DKB200: (EMU2) DATA2 R 0.00 0.28 0.00 5.00 $4$DKB300: (EMU2) DATA3 R 34.00 24.63 0.00 50.33
So, basically, a disk can be hot-swapped into the DS25E (using the on-board motherboard controller) but a reboot needs to be scheduled for everything to get back to normality.
A partial solution, until the reboot can be scheduled, and to reduce the slowness of a backup to tape is to change the read_cost of the disks that make up the shadow set to bias them towards the remote server.
Thank you to Volker for his invaluable help and partial solution/interim work-around.
- « Previous
-
- 1
- 2
- Next »