VMS Poor SDLT performance

[Tom O'Toole]

Gunther, yes I'm using vms backup for all these tests. The input disks for these tests are snapshot VDISKS on an eva5000, are not fragmented, and generally contain just a few container files. I don't think input is a botteneck (however I will try backup/phy when I get a chance). For this particular data, we see a single drive go full out at 20-25MB/sec (these measurements are obtained from the switches). In the case of four libraries (eight drives) on a single MDR, we are basically getting 50-55MB/sec, so that's only about 6-8MB/sec (and it shows, believe me). From our testing, this appears to be the internal limit of the MDR unit.

Anyway, we can drive a single drive (either SCSI or fibre) at full blast, but running both a libraries drives off the DAS results in severe degradation, while running both a libraries drives via the MDR result in minimal degradation. In both cases a single SCSI cable is used to connect both drives (and the robot) the card/router. The MDR has four connections for scsi, and we have also played with connection each drive to a different port, but we didn't see any noticeable difference from this configuration. The most interesting thing is that, unlike the directly attached SCSI card, the MDR doesn't seem to have the same difficulty running two drives on the same bus/cable (and thus we are loving the MDR).

Oh, I should also mention we are using multiple hosts w/ two 2G fc2384 HBAs for this testing, basically one host per library.

Thanks, Cass, Uwe, and Gunther for your comments.

[Antoniov.]

Tim (Nelson),
just for curiosity, your backup rate is changed? Any news?

Antonio Vigliotti

[Tim Nelson]

Antoniov,
Nope. Nothing changed.

I do see others looking to have the same problem in this thread.
2 drives on MDR = 1/2 the speed on both.

BACKUP command needs to be double buffered or at least be able to continuously fill the buffer as it is emptying.

Basically the BACKUP command or VMS in general cannot keep up with the new technology ?

[Guenther Froehlin]

Tim (Nelson),

this:

"Basically the BACKUP command or VMS in general cannot keep up with the new technology ?"

is a vastly speculative conclusion. As I mentioned before: Try a BACKUP/PHYSICAL/BLOCK=65024 with 1,2,3,... parallel streams. The physical backup is a) truely double buffered and b) does a spiral read and so does the fastest possible read from disk (besides the limited block size). This test will show you where the hardware bottleneck is.

[Tom O'Toole]

Tim, we have strayed from the original topic (but with much interesting discussion) As I recall your problem was a CPU and input disk related problem (your CPU was >80% and your input disk queue was high). Double buffering the output will not help this. Defragmenting the disks, raising the priority of the backup job, getting a faster cpu, these will help. Once your CPU utilization approaches 100%, you could turn off /crc to speed it up some, but after that there ain't no more in the tank.

[Tom O'Toole]

Here's a followup re: MDR performance. We have just gotten in an NSR with the four SCSI LVD port module. The NSR is running eight SDLT320 drives flat out streaming. Wow!

[comarow]

We are seeing that on SAN devices that the cache thrashes with diolm even at 100. We are recommending that diolm be set down to 32. That might be your case if your device has it's own cache.

Bob

[Jan van den Ende]

Bob,

please give any (as much as you can) specifics.
We are NOW in Nashua, (only for a few days) so NOW is the time we can get kicking!

We formally may require ANY engeneer to attend (counting on you now Sue!), but, there NEED to be questions, and they NEED to be concise!

fwiw,

Proost.

Have one on me.

jpe

[Uwe Zessin]

Of course the cache thrashes!!

A backup typically does not read data twice!

A /VERIFY does not count, because that is after the save set has been written and unless the set was very small the cache's contents have been purged a long time ago.

[Antoniov.]

This thread is hard to die!

Antonio Vigliotti

[Uwe Zessin]

Tim can close and reopen it any time.

[comarow]

There is a new document from engineering:
Setting Software Parameters for Efficient Backups


I don't know what your backup device is connected to. I would suggest setting diolm to 32.

[Uwe Zessin]

Bob, are you willing to share that URL with us?

[Dave Gudewicz]

Bob,

Please share the location of this new backup document.

Thanks,

Dave...

[comarow]

I'm not sure it's a public document.

If you can see it, it's public

http://h71000.www7.hp.com/doc/82FINAL/aa-pv5mj-tk/00/01/117-con.html


Good luck. The basic idea is everything we did do speed up backups gets turned on it's head with the cache of new devices. The old philosphy was to pump out as many disk queue lengths as possible to the device. That is no longer the case.

[Galen Tackett]

> http://h71000.www7.hp.com/doc/82FINAL/aa-pv5mj-tk/00/01/117-con.html

That's a link to a section of the same V8.2 system manager's manual that you can also find at http://h71000.www7.hp.com/doc/os82_index.html (though citing it here was a good idea.)

A comment on the text of that section--


You can obtain the best performance by placing files and all their extents on disk in alphabetic order and make the files contiguous. You can accomplish this by using BACKUP/IMAGE when you do an image save-and-restore.


Though this wording a bit ambiguous, I take it to mean that you should use /IMAGE on both the backup and the restore.


Or can someone show me a way to "do an image save-and-restore" WITHOUT "using BACKUP/IMAGE" on the save side? :-]

Also, does a /IMAGE restore actually put "[the] extents on disk in alphabetic order?" If so, that's a pretty cool trick though I'm not sure how to alphabetize disk extents. :-]

[Uwe Zessin]

Big yawn. That's a refinement of the links that Dave Gudewicz and Wim Van den Wyngaert posted in february. Re-read this thread and you will find out that me and others have already challenged some myths from the past.


I really would have expected something more of substance in this document and not such platitudes like:

> You can speed up backup operations twofold or
> more by replacing all or some of thes hardware
> components with ones that perform faster.

Oh! Really??

> Therefore, how files are laid out on disk is important.

That's great if you have a single disk, but on a virtual RAID system you cannot control this.

> Fragfmentation can also slow down BACKUP considerably.

Well, at least somebody has learned after many years...

[Ian Miller.]

the key thing to note from that document is the stuff about quotas especially that setting them really big is not the right answer.

The rest of the document is standard stuff.

[Jan van den Ende]

You can accomplish this by using BACKUP/IMAGE when you do an image save-and-restore.


:-)

So, now I am only missing a pointer to the instructions of how to do that without taking that disk out of production. :-(

Proost.

Have one on me.

jpe

[comarow]

While the new guidelines suggest reducing diolm to 100 or 32, we've seen cases where it works best as low as 4 or 8.

Your mileage may vary.

It certainly changes backup parameters on their head.

Good luck.

[Antoniov.]

It's really true document has no news for this trouble. However, because this thread is very long and Comarow is newbie here, I appreciated his work.
Cheers.

Antonio Vigliotti