StoreEver Tape Storage
1748117 Members
3697 Online
108758 Solutions
New Discussion юеВ

detect shoe-shining

 
SOLVED
Go to solution
RalfG
Frequent Advisor

detect shoe-shining

Hi,

I'm backing up large amounts of data to a lib with 2 LTO-4 drives. Because of the long time the backup takes to complete, I don't want to spool the data to disk and then write to tape for full backups.

The disk storage (file server) has a seq. read performance of 200-300 MB/s and is connected by GbE to the backup server.

The data are mainly large files.

The following is the output of dstat on the file server during a full backup with spooling enabled.

--dsk/sde-- --net/eth1-
read _writ|_recv _send
104M 0 |2509k 105M
104M 0 |2541k 106M
104M 0 |2536k 106M
72M 0 |2032k 85M
0 0 | 12k 1848k
58M 0 |1035k 42M
102M 0 |2527k 105M
106M 0 |2558k 107M
104M 0 |2543k 106M
107M 0 |2548k 106M
106M 0 |2558k 107M
106M 0 |2555k 107M
104M 0 |2558k 107M
16M 0 |1413k 60M
0 0 | 11k 1313k
0 0 | 70B 70B
64M 0 | 295k 12M
104M 0 |2555k 107M


It looks similar without spooling. I'm not sure what the reason for the short dropouts is.

I'm a bit worried about tape shoe-shining without spooling and want to know if there is a way to get the status about how often a drive had to start/stop during a full backup (scsi command, ltt...).
18 REPLIES 18

Re: detect shoe-shining

Shoe-shining doesn't really happy on modern LTO tape drives. LTO has always supported something called DRM (Data Rate Matching), which allows it to adjust the speed of the tape according to the rate of data being streamed in. LTO4 for instance will adjust its speed between 40MB/s and 120MB/s to present shoe-shining.

So in-short - don't worry about it.

HTH

Duncan

I am an HPE Employee
Accept or Kudo
RalfG
Frequent Advisor

Re: detect shoe-shining

I know that LTO-4 adjusts the speed. But if the data stream drops below 40 MB/s for a short time (as seen in my dstat output above) doesn't the drive has to stop/start write again? I think the drive has a 128 MB buffer which will help, but if the data comes too slow (eg. incremental backups) shoe-shining can happen. Or am I missing something else?
Marino Meloni_1
Honored Contributor

Re: detect shoe-shining

you can got some indication in the support ticket.
You can also test (with LTT) where the problem is:
In LTT there are several performance test available: system performance and device performance.
first use the device perfomance, this test will generate data in the host memory and send it to the LTO drive, this will give you some indication about the scsi subsystem, yo should be able to reach the teorical tape throughput
then you have the system perfomance, here you can test the the source of the data, and have an idea of the max throughput your system can deliver, you can also create a fixed amount of data (this is the restore test) and then test the system with those data, and this will give you an idea if your problem is comming form the HW or from data (defragmented, openfiles, antivirus....)
RalfG
Frequent Advisor

Re: detect shoe-shining

I have performed various test with ltt, hptapeperf and dd. The general performance is fine, I can write with ~60-80 MB/s to tape (real backup, highly compressed data, single client).

With dd and the other tests, I get read/write performance up to 190 MB/s.

The problem is, I don't know if there are periods during a backup with an average throughput of 70 MB/s, where the throughput drops below 40 MB/s. I can see with dstat that there are periods where the the network throughput between file server and backup server drops to <10 MB/s, but I don't know if the drive starts shoe-shining then.

If this happens only a few times during a long running backup, it'd be fine. But if this happens often, I'm worried about the additional stress for the tape and drive.

Hence I would like to see some number on how many time the drive had to stop/rewind/start during a given time.

Maybe I'm too afraid about this. But sometimes people forget to put the data in gzip format on the file server, the result is many very small files...
Marino Meloni_1
Honored Contributor

Re: detect shoe-shining

well LTT record the info you need, but you should contact HP in order to extract these info from the EEPROM data of the support ticket as it is not accessible normaly

see example:

|__ Mechanism EEPROM Page
||__ Power on time : 109243379
||__ Power cycles : 88
||__ Tape pulled : 30012154
||__ Pulling time : 12134187
||__ Load/unloads : 1595
||__ Start/stops : 1295420
||__ Thread cycles : 3537
||__ Repositions : 7032
||__ Turnarounds : 1219383
||__ SDRAM errors : 1
||__ CRC errors : 0
||__ Cleanings : 16
||__ Upgrades : 3
RalfG
Frequent Advisor

Re: detect shoe-shining

I see that the relevant code is in scripts/LTO_ReadLogs.sts. There is no way for me to get this info without HP's help?
Curtis Ballard
Honored Contributor

Re: detect shoe-shining

If you really need to get to the information we can tell you how to extract the reposition information from the logs without using L&TT but most of the time there isn't any real need to figure out if the drive is repositioning if your backups are fast enough to satisfy you.

There are two things that happen with Shoe-Shining.

1) Extra mechanical wear
2) Performance impact

With HP LTO tapes and drives both the drive and the tape cartridge are extensively tested and required to meet standards where the extra mechanical wear isn't an issue. You will get the rated life in worst case operation.

2) Performance impact

Here HP's LTO tape drives really shine. We have the variable rate matching which isn't available in some other LTO tape drives. With variable rate matching the tape drive will slow down as the data rate slows down up to 1/3 of the max speed. The drive input buffer is designed such that at 1/3 of the max speed the buffer is large enough to hold all of the data that can come in during a reposition. That means that the drive can always write the data at the speed it arrives. Something not every LTO drive can do.

The specific log page you will need if you want to see the reposition information is 34h. If you have a way to issue the SCSI Log Sense command and get back a log page 34h I can provide the information on what field has the reposition information.
Tom O'Toole
Respected Contributor

Re: detect shoe-shining


What kind of library do you have. On the MSL5000/6000 series, it tells you what it's doing on the front panel status display. You can spot possible shoe-shining if it's changing from writing... to idle... repeatedly.

It seems like the breaks you are seeing could just be an artifact of buffering on the host file system, and maybe your samples are too frequent to smooth these out. You don't say what the interval is.

Assuming this is linux, you could maybe try piping through the 'buffer' program and see how that goes.
Can you imagine if we used PCs to manage our enterprise systems? ... oops.
Cass Witkowski
Trusted Contributor

Re: detect shoe-shining

I have not looked at the capacity in a single pass of the LTO-4 tape cartridge. Be aware that when you write to the tape it actually writes using several head through the length of tape then moves the heads and writes data going back the other way. It can do this many times. Whay you may be seeing is not show-shining but actually the the LTO-4 tape reaching the end and then switching to write the other way.