topic Re: Relationship between BACKUP qualifiers, /CRC and /GROUP in Operating System - OpenVMS

Relationship between BACKUP qualifiers, /CRC and /GROUP

Clark Powell — Fri, 16 Sep 2005 16:00:53 GMT

On the BACKUP command, if /NOCRC is set then is the /GROUP qualifier irrelevant? We were using /NOCRC and then added /GROUP=0 but saw no increase in performance. One opinion was that since we were already using /NOCRC that the /GROUP qualifier was irrelevant. Is this correct?

wcp

Re: Relationship between BACKUP qualifiers, /CRC and /GROUP

David Jones_21 — Fri, 16 Sep 2005 16:24:06 GMT

The CRC is used to detect data transfer errors writing or reading the tape. The redundancy group adds parity data to let you recover from one bad block within the group.

Re: Relationship between BACKUP qualifiers, /CRC and /GROUP

Ian McKerracher_1 — Sun, 18 Sep 2005 07:49:32 GMT

There are a number of HP manuals, journals and ECO summaries that explicitly state both the /NOCRC and /GROUP=0 qualifiers in the same Backup command. I have also seen examples that use the /NOCRC and /GROUP= qualifiers in the same command. I think we can assume that the CRC and GROUP qualifiers are not related and that /NOCRC/GROUP=0 is a meaningful combination.

Regards,

Ian

Re: Relationship between BACKUP qualifiers, /CRC and /GROUP

Volker Halle — Sun, 18 Sep 2005 10:40:24 GMT

wcp,

/CRC and /GROUP_SIZE are 2 different mechanisms to protect against data corruption or loss in backup savesets.

/CRC creates a CRC checksum over each backup block being written. Using /NOCRC does not create (when writing a saveset) or check (when reading the saveset) the CRC. This saves some CPU cycles, but does not protect against detecting bad data in backup saveset blocks.

/GROUP=n writes a redundant backup block every n backup blocks, which allows recovery of ONE uncorrectable read error inside each redundancy group. Using /GROUP=0 will reduce the no. of backup blocks written by 1/11th (default is /GROUP=10). You should be able to see a slight reduction in the DIRIO count. If the input disk is the bottleneck, you won't see any performance increase.

Volker.

Re: Relationship between BACKUP qualifiers, /CRC and /GROUP

comarow — Sun, 18 Sep 2005 12:50:14 GMT

It has been recommended with modern backup devices to set group=0
However, while many don't use CRC, CRC is software to software checking. It does consume CPU cycles. It is your choice but the reliability of backups is improved with CRC checking.

Please see the new standards for tuning backup.

You should note that diolm with SAN devices should be reduces to 100. Below 100 Xwindows won't start, but if you are running batch backups you can reduce it to a much lower number, such as 32. There is a problem with Cache thrashing. Engineering is aware of the problem. But for better performance reduce your diolm.

Here is the new backup recommendations, and those are conservative so windows will start.

http://h71000.www7.hp.com/doc/82FINAL/aa-pv5mj-tk/00/01/117-con.html

Re: Relationship between BACKUP qualifiers, /CRC and /GROUP

Wim Van den Wyngaert — Mon, 19 Sep 2005 06:23:34 GMT

I would re-verify your test result.

/group=0 will decrease the traffic to your drive with 10% and cpu load on VMS will be lighter because no raid-like computations must be done.

We saw the differce very clear.

Wim

Re: Relationship between BACKUP qualifiers, /CRC and /GROUP

Bart Zorn_1 — Mon, 19 Sep 2005 08:18:00 GMT

I am not so sure if you can do entirely without /GROUP redundancy. Although no doubt modern tapedrives provide the necessary redundancy, /GROUP may be helpful if tapes are getting older and are possibly not stored in optimal conditions. In this case BACKUP may be able to restore valuable files when some blocks on the tape are no longer readable.

Of course, it is rather difficult to test this, but it is my belief that that I prefer to have the redundancy blocks and never need them than the other way around!

YMMV,

Bart Zorn

Re: Relationship between BACKUP qualifiers, /CRC and /GROUP

comarow — Mon, 19 Sep 2005 10:04:55 GMT

The documentation specifically recommends
/group=0

Re: Relationship between BACKUP qualifiers, /CRC and /GROUP

Wim Van den Wyngaert — Mon, 19 Sep 2005 10:11:06 GMT

Because all modern drives have a some kind of /group=4 built in. And when recovery is not possible, data is not returned to the backup process so backup can't recover. Only "modern" is not defined ...

Wim

Re: Relationship between BACKUP qualifiers, /CRC and /GROUP

Jan van den Ende — Mon, 19 Sep 2005 11:03:32 GMT

Define "modern" as "SCSI", and any recovery processing is simply IMPOSSIBLE because the first parity error found will simply ABORT data transfer. Unrecoverable. Period. (unless you are able to locate a DSA compliant tape for the tape format).

SCSI = Scuzi = Sorry for this functionality-lacking implementation.

fwiw,

Proost.

Have one on me.

jpe

Re: Relationship between BACKUP qualifiers, /CRC and /GROUP

Tom O'Toole — Wed, 21 Sep 2005 11:24:42 GMT

There is a thread where this has been discussed at great length, including the latest recommended values - it's not clear that they are absolutely correct:

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=658938

Re: Relationship between BACKUP qualifiers, /CRC and /GROUP

comarow — Wed, 21 Sep 2005 13:45:04 GMT

Here's what is actually in the documentation.

Table 4 BACKUP Qualifiers That Affect Performance Qualifier Description
/BLOCKSIZE
To write a save set to a tape device, always use /BLOCKSIZE = 65,024, the largest block size you can use with save sets on tape.

Note: To be able to copy save sets from tape to disk, use a maximum block size of 32,768.

/GROUP
Today's tape and disk drive technology makes the XOR group feature of BACKUP obsolete. Use /GROUP=0. (If you do not specify the /GROUP qualifier, the default of 10 is used, which adds 10% more data to a save set).

/CRC
Keep the default of /CRC, which does not add extra data to the save set. A 32-bit field is always reserved in a save set whether you enable CRC or not. However, adding a small amount of CPU time helps to ensure the integrity of a backup save set.

Re: Relationship between BACKUP qualifiers, /CRC and /GROUP

Ian Miller. — Thu, 22 Sep 2005 03:06:55 GMT

I belive a slightly smaller block size (multiple of 2k or 4k - I can't remember) is recommended when using FC storage.

The big change in the new recommendations is to have a smaller DIOLM when using modern storage.

Re: Relationship between BACKUP qualifiers, /CRC and /GROUP

comarow — Thu, 22 Sep 2005 03:15:07 GMT

Absolutely correct. A high diolm causes the cache in the San controllers to thrash.

However, engineering is aware of the problem.

While interactive jobs require a diolm of 100, if you have batch jobs, not just backup, but other i/o intensive operation try reducing your diolm to 32 or even less.

Look at the accounting report for results.

Bob

Re: Relationship between BACKUP qualifiers, /CRC and /GROUP

Wim Van den Wyngaert — Thu, 22 Sep 2005 03:40:58 GMT

Can someone of HP confirm that all SCSI DLT drives have some kind of raid protection ?
E.g. TZ8* ?

Wim

Re: Relationship between BACKUP qualifiers, /CRC and /GROUP

comarow — Mon, 26 Sep 2005 20:20:17 GMT

A customer of mine did some testing with and without CRC testing.

He did do the test with the TL891 tape drives

Backing up a 20,000,006 block file (512 Byte blocks) I got the following results

Blocksize = 32256

WSQuota = 16384

Compression - off

CRC GROUP Elapsed (seconds)

Y 10 2328

Y 10 2305

Y 10 2407

Y 0 2205

Y 0 2172

Y 0 2172

N 10 2370

N 10 2399

N 10 2396

N 0 2202

N 0 2168

N 0 2167

The biggest gain is from turning off Group (i.e. group=0). This causes less tape to be written.

I guess that puts the issue to bed!

Bob Comarow