Harry D Brown Jr! (re: Centera)

[Jim Mallett]

This is for anybody using an EMC Centera, but from reading the forum I see that Harry has had to deal with some issues that I'm going thru.

My average file size is only 50k on our Centera, but there are currently 7 million files. I'll probably top out at 10 million files. Retrieval is slowly deteriorating. I'm using Legato's DiskXtender as the gateway and I think the problem is there with their virtual media.

Does anybody have experience with the Centera and many small files?

Thanks....
Jim Mallett

[Stuart Whitby · Accepted solution]

Apologies for the delay on this - just noticed this when searching for Centera +"small files" +performance on google. This could be down to 4 things:

read and write contention: Are you migrating data at the same time as you're trying to read it? I believe there's a registry key which can be changed to help with this, but can't remember what it is offhand. Trying to find out from someone more familiar with DX than me...

Performance problem relating to queueing of fetches within DX: How many fetches are you doing per day? If it's a reasonable number, this is likely to be increasing as more files are migrated, and you may be seeing the results of fetches sitting in a queue for a while before they're fetched from disk. I'm unsure if fetches are done in a single-threaded manner (if so, it'll probably be a single thread per piece of available media).

Performance problem querying Centera: How much media do you have online? It takes a long time for DX to fetch this from Centera, and this may be causing you problems. Change the registry setting "HKEY_LOCAL_MACHINE\SOFTWARE\Legato\XMS\Service\00001\Refresh Rate" to 3600 to update this once per hour - it doesn't really require frequent updating. Can't remember whether this is a DX or Centera problem.

Performance problem with the machine running the Xtended drive: Is this DXUL or DX2000? My main experience is with DX2000, but I've seen problems due to the number of open files on that system. High numbers of files being opened per second, and open files ranging from 3000 overnight to 13,000 during the day. Every file opened now has to go through two accesses under DX' spy driver - one to check whether the file has been migrated and another to open the file for the calling application. However, this problem would be apparent from the start, and not from 7million files in - unless you're now getting queued fetches, as above.

[Jim Mallett]

Stuart,

Thanks for the reply. I ended up working with Legato for quite some time on this and put in a couple of workarounds, one of which being the refresh rate you mentioned.
I am using DX2000 and the problem does relate to that software and the number of files on each piece of virtual media. They state they will have a fix in their next release at the end of the year.
The other workaround was pre-fetching 45 days worth of files and keeping them on the gateway server. (This somewhat defeats the purpose of the Centera but as long as I'm not listening to end users I'm happy for now.)
You were right on the money regarding the refresh rate though.

Thanks....
Jim