Ultrium 960 - Fails when reading data ("Unknown Error")

[Fenixhawk]

Had this tape drive for 2 years without issues, but last week it started failing on random tapes (which are known good.) It would seek for roughly 20 minutes, then stop with a "Unknown Error". Now, I'm either getting "Unknown" or "CRC32 Error". I can still read data from some tapes, but the problem seems to be progressively getting worse.

Internal Ultrium 3-SCSI Firmware: D21H (OEM Model)
Model PD003C#800, Rev G003
Part no. 973604-104
Connected internally to Adaptec 29320LPE (PCI-e) SCSI Card

LTT:
"There were unrecoverable read errors while reading tape 2060823248 (ÿÿÿÿÿÿÿÿ)."
"A checksum error has been identified in the drive electronics."

I can still write to tapes, catalog, erase, etc without issue. But when trying to read data from the tapes, it's consistently failing. (Log file is no help.) Cleaning the drive doesn't seem to help either, and I ruled out heat and cabling as an issue.

I contacted HP Technical Support via email (couldn't get through to anyone on phone), but it's been a few days since any kind of response. And the big problem now is that my warranty has just run out.

Is there still a way to extend the warranty, or get a RMA? (HP Service Packs? How much do they cost?)

Does it sound like the drive head is failing?

[Fenixhawk]

Correction: "HP Care Pack service", not "HP Service Packs"

[Richard Bickers]

It doesn't sound too healthy given what LTT is reporting.

Best test we have is the LTT LTO drive assessment test if you have a tape you feel is in good condition you can use. This tests the drive at all speeds across different areas of the tape and in both directions. Gives a pass/fail response - though may ask you to try a second tape just to be sure it's the drive.

Perhaps before doing that, pull a support ticket and post it to this thread and I'll take a look at the internals.

It may have got gunked up and a clean is what we'd normally recommend. Try a couple before giving up. If it's really beyond warranty (and the only alternative is a replacement) then try an alchocol swap over the heads which you can just about reach through the door. Be very careful not to apply pressure as the head alignment is sensitive and you can't fix it if it moves.

Any chance it could have become contaminated? Any nearby building work for instance? Beware air con vents blowing air though underfloor spaces and straight into the drives.

Good luck.

[Fenixhawk]

Yeah, I had tried the assessment test on 3 cartridges (all brand new: 2 HP and 1 Quantum model). It failed all 3 times on the 4th pass. I've attached the Results + Support Ticket.

Thanks for the cleaning tip, I had only cleaned it once but I'll try again after you examine the diagnostic files. I tried to avoid using the tape cleaner more than once, because I heard that overuse could strip the head overtime. I want the drive to last as long as possible, it's one of the best investments I've ever made. :)

As for contamination, nope - it's a very clean environment with hardly any dust.

Good news though, I tried a couple more times to reach a technician by phone and finally reached sometime. Looks like the Warranty Lookup tool was reporting the wrong the expiration date, and I still have a month left.

If a replacement is in order, do they come with a new warranty by chance? Or would my only option for future repairs be a Care Pack?

Thanks much!

[Fenixhawk]

Attachment for the second file (Assessment Log File)

[Richard Bickers]

Your ticket shows a drive in poor condition I'm afraid. Multiple heads (3 out of 16) struggling. It's unusual to see this without some form of contamination (sometimes I see a single head down for instance) so I'd look really carefully at your environment. As this has happened suddenly I'd suspect something along the lines of nearby construction. 'Normal' dust is fine as it's organic - it's the abrasive particles that can do the damage. Also nearby walkways or printers can be a problem.

Don't worry about over cleaning. The cleaning tape is not *that* abrasive and, frankly, your drive needs a good scrub anyway. There's a little brush in the drive that is activated when the cleaning tape goes in which can help too.

Good news you still have a month of warranty left. Best not stick a swab inside your drive just yet then. You'll probably get another drive OK (just tell them what you did with LTT) but it won't extend your warranty... Maybe ask them about the care pack on your call.

Glad you like the drive - we're very proud of them. Just a shame this one has gone south - it's not that common. I still suspect your environment.

[Fenixhawk]

Alrighty, I've received my new tape drive. My old one was a half-height (5 1/4" size) drive, and the new one is full-size (3U) Ultrium 960. I have a concern about the internal temperature however. On my old unit, it never got hotter than 53 degrees Celsius (considered "Excellent Margin"), however this new unit runs on average at 73-75C. System is the same, room is kept very cool.

When trying to restore tapes earlier today, it reached 98.3 C on the first tape!

Is this normal? Anything I can do to keep the temperature down, configuration or otherwise?

I'm also having problems restoring data from two tapes (CRC32 Error on one, and "Unknown Error" on the other - just like before on the old drive). Could the old OEM tape drive (with the 3 bad heads) have caused damage to the tapes? Will using these bad tapes cause any damage to the new tape drive?

[Richard Bickers]

Even though LTT is reporting internal temperatures that does sound too hot. What does the ticket say? As long as it doesn't say 'Warn' you should be OK but you'd want a bit more margin than that for peace of mind.

Airflow is the main reason for drives to run hot. Is it an external (in a box) or internal drive? The external comes with a fan but for the internal, it's up to you to make sure it has good airflow. Same as any other internal device.

You can use the LTT temperature test to measure the temperature at operating speed (it runs for 30 mins).

Your tape errors are consistent with what you had before which suggests the issue is with the tape. The drive can't really write with 3 heads down so should have failed on the backup. If it was OK at the time of the backup then these tapes should read fine with the new drive.

What is the history of the new drive? Doesn't sound like a replacement from HP as you'd get like for like.

Usual troubleshooting at this point is to pull a ticket. That tells us quite a bit about the drive. If you post to this thread I'll take a look.

I'm a bit worried about your tapes though. How critical is the data?

[Fenixhawk]

I'll do some more tests tonight, but it did give me a warning in LTT when it reached those high temperatures. I tried sitting the tape drive outside of the server, but it's still getting pretty hot. (From an off-state, I'll boot up the PC and restore a tape right away. It gets hot fast within 20 minutes.)

I'll try the LTT temperature test as you recommended. I did a media analysis on one of the tapes, first time it failed, second time it was successful.

Both tapes were good for a long time, was able to backup/restore to them without issue. But then they started having problems when my original drive started failing randomly.

One tape I can recover over a few weeks, but the other tape had some critical information on it. Is it possible that dirty heads on the previous drive could have contaminated the tapes in any way? Or possible heat damaged the tapes?

(Re. history of the new drive - LTT doesn't show any log files on it, could have been wiped before shipping if it's a refurb. My guess is that they don't carry those OEM models anymore, like my original one.)

[Richard Bickers]

To get the temperature right you do need airflow through the drive. Just sitting in a cool environment isn't enough when it gets going.

If the media analysis test was successful that's a good sign but you'll need to set it to do a full volume in the options. Perhaps try that (it's independent of any backup/restore app) and then pull a ticket at the end so we can see how much effort the drive had to put into reading it.

Tape contanmination is a possibility. We can see some of that from the above process. Unlikely heat damage though. That would reduce the ability of the drive to read/write the data but not actually damage anything.

HP doesn't do refurbs (just warranty or care pack replacements) so I'm a little uncertain of your new drive but the ticket will tell us how well it's performing with your data tapes and you can try the assessment test with a new (or good) tape to verify the drive is in good shape.

[Fenixhawk]

I ran a media analysis test (full volume selected), and it came back perfectly fine - no issues. It appears that the one tape is fine, but I can't read any data from it. I can run a catalog, but selective restores always end up with a CRC32 error.

The data was backed up using "Genie Backup Manager 8.0", so I can't retrieve it using BackupExec or other tape software. Have you heard of any free applications that allow a full volume tape dump, without requiring a catalog? Something that just reads the entire tape into an image format for example.

There's "TapeDump", but the trial only allows 25MB of data for testing (can't be accessed).. so as far as I know, it might not work if the full version is purchased.

[Richard Bickers]

The data seems to have been corrupted on the way into the drive which is about the worst thing that can happen. These drives wrap their own CRC around every block/record of data as it arrives and it's formatted with it when it is written to tape. If the drive is reading back the full tape volume without error then every one of those CRCs must be intact as they are checked on the way out. Even though the drive might have been strugging to write the data physically, it seems to have done so without error - which is what they're designed to do.

Only other place the corruption could occur would be over the interface itself unless there was a software glitch which seems a bit unlikely. SCSI has parity detection but it's not foolproof.

I'm not aware of any software for recovering tape data. It depends so much on the format of the back up application and we usually suggest users go back to the application vendors to see if they can help. Maybe Genie has an 'ignore errors' mode for instance. Might be worth getting in touch with them first.