Email Subscription Notifications Suspended Temporarily
We are in the process of making navigation in the Servers and Operating Systems forums simpler and more direct. While doing this, we have to temporarily suspend email notifications for subscriptions. If you are subscribed to one or more discussion boards or blogs in the community, please check them daily to see new content. Notifications will be turned back on in a few days. We apologize for any inconvenience this may cause. Thanks, Warren_Admin
Technical Support Services
Showing results for 
Search instead for 
Did you mean: 

DPTIPS: 1 way to ensure uninspired StoreOnce VTL performance and 7 ways not to

Jim Turner

It's a secret I reveal to precious few. I dare not share it carelessly. But if you promise not to run around all willy nilly shouting my mystic formula hither, thither, and yon, I will impart to you the complex series of steps necessary to extract mediocre performance from your StoreOnce appliance.


This is a little hard to digest all at once, so be prepared to read it through again.


Are you ready?


1. Nothing.


That's right, nothing. Don't follow best practices for integrating Data Protector with StoreOnce VTL. Don't create more than one VTL based upon data types. Don't disable multiplexing. Don't use enough streams. Don't use the correct cartridge size. Don't make your media pools non-appendable. Don't force Data Protector to overwrite expired cartridges before touching blank ones. Don't license your StoreOnce appliance by capacity rather than drives and slots. Don't do any of that. Just do nothing -- a single, easy step. One and done. Presto! Crumby performance and no effort.


Or not.


If you would prefer to enjoy maximum benefit from your StoreOnce investment, let's take a look at what you SHOULD do and how easy it is.


Multiple VTLs
Let's create one gigantic VTL and send everything to it. Simple, yes, but it's wrong. Here's why.


StoreOnce tracks duplicates within each VTL but not across VTLs. That's on purpose, and you should use it to your advantage. Why spend resources trying to find duplicates in data that holds very little promise of actually being duplicate? Your odds of finding a 4 KB chunk of Exchange data that matches a 4 KB chunk of Oracle data? Not likely. The odds of finding a chunk of Fileserver data this week that matches a chunk from last week? Much better.


Create one VTL per data type. If you are backing up Exchange, Oracle, Fileserver, Windows OS, and Unix OS data, I would create five VTLs with crazy names like EXCH_VTL, ORA_VTL, FILE_VTL, WINOS_VTL, and UNIXOS_VTL. Segregate your backup data accordingly. Each VTL represents a smaller dedupe domain so the hash comparisons are faster, and you'll likely find more duplicates which means the 4 KB backend write is avoided saving time and space.


And don't be afraid to create each VTL with 12 drives for reasons that will become clear in the following sections.


No Multiplexing
Also known as device concurrency, this is a metric that describes the maximum number of disk agents that will be allowed to send data to a target device at once. The default device concurrency in DP is 4 which is too low for physical tape and about 3 too high for virtual tape. To disable multiplexing, simply set the device concurrency to one (1) for every virtual tape drive. Now increase the number of virtual drives you have available and put plenty of objects in a backup. You've just replaced multiplexing with multistreaming.


Why is this important? Because multiplexing wrecks inline dedupe performance and diminishes the ingest rate. When you let DP slice and dice, commingling data blocks from multiple objects to one virtual tape, the StoreOnce appliance chunking is far less likely to find an inbound block that matches exactly a previously stored block. Now you have paid the price for inline dedupe processing without realizing any benefit in terms of space savings or ingest performance. In other words, a net loss. So don't multiplex with virtual drives. Device concurrency = 1. Always.


Sufficient Streams
Each virtual tape drive that DP sees is actually a small collection of Linux processes within the StoreOnce appliance pretending to be a tape drive. Each set of processes is capable of a finite number of I/O operations per second. Don't be surprised when a single virtual drive cannot outperform a single physical drive. The physical drive is not doing inline dedupe, but the virtual drive is. Fortunately, the StoreOnce appliance as a whole has enormous resources and can host many, many virtual drives running at the same time.


You should leverage that fact by ensuring that you have a sufficient number of streams (active drive connections) inbound to the appliance concurrently to realize its rated ingest. These streams can be coming from one or several backup specifications, and the objects involved are likely to be sourced from numerous clients.


Using our StoreOnce 4430 appliance as an example, you will need at least twenty (20) concurrent streams with no more than twelve (12) going to any one VTL before you can achieve the appliance's maximum ingest. But once you replace multiplexing with multistreaming, it's a cinch. Plus, you will achieve greater ingest (with dedupe!) than you had formerly with a handful of physical drives.


Cartridges Sizing
There is no penalty for not filling every virtual cartridge. Additionally, StoreOnce does not preallocate space based upon the size and number of cartridges you create. Finally, there are reasons (beyond the scope of this discussion) why it would be best not to span dozens of cartridges per backup.


Assess the size of every object to be written to a given VTL. You can see object sizes in the properties of previous backups. Based upon the size of the single largest object, select a cartridge size at the next increment. If the largest object is 650 GB for instance, select 800 GB as your cartridge size.




One object per cartridge per backup. Nice and clean.


Non-Appendable Media
StoreOnce VTLs are happier -- especially where low-bandwidth replication is in use -- when you start each backup with fresh tapes rather than appending to partially filled cartridges. Space used is space used. It doesn't matter to the appliance or to DP whether 800 GB of data is on 8 cartridges or 80. Plus, there is no incremental cost for virtual cartridges as there would be in the physical world.


Naturally, you will want to calculate how much media is consumed by backups utilizing a particular VTL, multiply that figure by the number of iterations you plan to retain, then provision an adequate number of slots in the VTL. Adding a few slots for margin wouldn't hurt, either.


Let's say you are creating a VTL for MS-Exchange backups. The weekly full consumes 6 cartridges and is protected for 10 weeks. The daily incremental consumes 2 cartridges and is protected for 7 days. How does the math work with that?


Weekly: 6 tapes x 11 weeks = 66 tapes
Daily: 2 tapes x 8 days = 16 tapes
Total: 82 tapes + 8 margin = 90 tapes


Oh, go ahead and call it 100. I like nice round numbers.


You might wonder why I bumped up the weeks and days in those calculations. It's because the protection begins at the end of the backup, but media is needed at the beginning of the backup. Thus the oldest media would not be quite expired when it came time to be used at the beginning of the next cycle.


Expired vs. Scratch
Given 100 freshly formatted tapes, DP will think it's doing a good thing by preferring unused media over cartridges that have been used before and are now expired. This will continue until all 100 cartridges have been written once, then DP starts from the top with overwrite #1 for each (expired) cartridge. In the physical world, this behavior is desirable because it guards against reusing a small subset of cartridges while others sit gathering dust. In the virtual world, this is the very last thing you want DP to do because it prematurely fills up your StoreOnce appliance.


Why? Easy. Expired cartridges still consume space on the StoreOnce appliance until they are reformatted or overwritten with new backup data. So unless you enjoy reformatting tapes each morning as a hobby, use one of these two simple methods for giving DP an attitude adjustment.


Starting from ... scratch? (No pun intended. Sort of.) Then don't format any of your virtual media. Seriously, don't. Instead, ensure that InitOnLoosePolicy=1 is set in your global options file. Scan all of your new virtual media so DP registers it as blank rather than unknown. Create the media pool to be used by that VTL, and start running backups using that pool. If insufficient expired media are already in the pool, DP will format enough blank media to satisfy current demand. If DP has enough expired media already in the pool, it will be used (overwritten) keeping StoreOnce appliance space consumption to a minimum.


Already have all your virtual media formatted? Don't worry, we can achieve the same behavior in a different manner. Create a free pool for each VTL. Reformat all of your expired media, placing them in the relevant free pool. Now modify the properties of your regular VTL pool so that it draws scratch media from the free pool but does NOT put it back there.




Voila! DP prefers to overwrite expired media already in the pool before drawing any scratch media from the free pool thus keeping StoreOnce appliance used space to a minimum.


Capacity Licensing
In the physical world, we license tape libraries by the number of drives and storage slots. Although you could continue to do that with virtual libraries, why would you want to? With DP's Advanced Backup to Disk licensing, you cover the physical usable space in your StoreOnce appliance -- 72 TB for instance. Then you are free to create as many VTLs with as many drives and slots as your needs dictate without purchase of any further licensing.


For each of your VTLs, be sure to let DP know that it's really a virtual library by checking the box indicated. Doing so switches license checking for that library from drives & slots to Advanced Backup to Disk capacity.




And we don't care if you cram 500 or 600 TB of user data into that 72 TB of physical space. The DP licensing is the same regardless of how much benefit you derive from inline deduplication. You're welcome. :-)



Though I've only hit the highlights, these are principles that I share onsite with customers nearly every week.  It's the same thing I taught in my Technical Online Seminar (TOS) sessions two years ago.  They are in many respects a summation of guidelines documented in our best practices white paper for Gen 1 & Gen 2 D2D appliances as well as Gen 3 StoreOnce appliances.  I have proven the accuracy of these principles time and time again in numerous customer environments, and I can do the same for you.


If you would like an indepth assessment of your backup and recovery environment and a week of intense, customized knowledge transfer, contact your HP Services Sales Representative and ask about HP's Integration & Technical Services team.


Tell them Mr_T sent you.  :-)



Was this information helpful? If so, please like and give kudos. Thanks!
About the Author

Jim Turner

Jim is a multi-disciplined engineering professional with 30 years of electronic and systems experience. For the past 17 years, Jim’s primary focus has been enterprise backup, recovery, and archiving (BURA). As an HPE Master Technologist, he is recognized as a global authority on HPE Data Protector, HPE StoreOnce, and the proper integration of both. Jim's consulting has stretched over 370k miles and 151 unique locations in North America during his 9 years with HP(E). When not traveling, Jim resides in Edmond, OK with his wife and two dogs.


I will refer back to this, it's a great summary. Cheers!


Hi Mr T


Great Article and long overdue! (Yes I know there is a white paper but getting customers to ingest these can be as hard as getting them to read the product announcements....)


I disagree a bit with your comment on not mixing data types in the same VTL. Totally agree that this is important for large businesses where backup & restore performance must be maximised but for smaller businesses where long-term data retention is key and the size of the backup device was dictated by the required usable storage to hold the backup data (e.g. length of time they want the backup data available online) sometimes having everything in one big VTL can improve this.


For example, I have a customer that needs to retain all his weekly backup data for 2 years online if possible. Most of his restore requests are for data minimum 6 months old. He had best practice config where each data type was in its own VTL and he ran out of space on the D2D4324 (G2 D2D). The cost to upgrade (have a 2nd device) was pretty high and as the full backup of the env is only approx 20 TB this a relatively small customer. We decided to forgo performance and combine data types into a single VTL and the device is only 52% full now, the dedupe ratio is 27:1 and growing and he is still totally happy with the backup & restore peformance, although a bit slower than before it is still well within his SLAs and he can now keep everything online for 2 years to meet his online data retention requirements.


I see a lot of smaller businesses buying these devices to suit online data retention requirements / to negate physical media management; and separating the data types doesn't always get them the dedupe ratio they need, a lot of them are investing so they can do away with physical tape, not for backup & restore performance.


So while I agree with your article in principle I do think that this isn't always the best way to go for smaller businesses and you'd be surprised at the actual overlap in identical blocks between data types provided your backup & restore SLA can accomodate the drop in performance?


I guess there is a lot to be said for simplicity, a lot of SMB customers just wanna replace their physical tape library with a virtual one, they don't care about the performance they just want it simple. I spend a lot of time trying to convince them that the performance is key but all they see is complications and additional config. I'll certainly be showing them your article as its a lot more succinct than the best practices whitepapers so thank you very much for taking the time to write this for us Smiley Happy




P.S. Are there plans to write a similar post for catalyst store & usage in DP? *Lots* of head scratching with my customers on the whole gateway can open multiple MAs but MA<->DA conccurency remains 1, so something around the useage of min/max devices to dicate the number of objects backed up concurrently in the session would be really useful..... And maybe DP object copy with replication enabled? Or am I asking too much now Smiley Wink

Jim Turner

Good morning Jenni,


Thank you so much for taking the time to post your wonderful comments.  It is the ability to correlate emperical data from different practices that helps us all.  It looks like you've found a usage case where one size does not fit all.


If they are seeing a 27:1 dedupe ratio, that would imply that on average only 1 in 27 chunks coming in are unique and being passed through for a backend write.  The other 26 are being tossed and replaced with pointers since they are duplicates.  That being the case, the performance should be pretty good.  In fact, it would only be diminished if they were pushing less than 20 concurrent streams at the 4324.  Well that plus the larger dedupe domain and its overhead.


I will definitely keep in mind your experience and factor that in to future scenarios.


Thanks also for your suggestion on future topics.  They are definitely worthy.


Best regards,




I have a client who lost the license of the VLS B6200. There is a way to get this license by HP.





Jim Turner

Hello Italo,


First, I presume you mean a StoreOnce B6200 rather than a VLS 6000 series appliance.  The latter is quite mature and has been displaced in large part by our StoreOnce appliances.


Second, your client should be able to contact HP with the serial number of the B6200 and retrieve previously issued licenses.  Please visit for more informaiton.




Chris Burcham

Any chance you have a copy of the best practices for the Gen 1 & Gen 2 D2D appliances? I've been trying to find that for quite a while, and every link I go to, comes up with nothing.  Thank you, and the article was very informative.  I was told differently at the initial setup of our D2D 4112's, and now performance and dedupe is not so great.  Also, target appliance is not able to keep up with housekeeping, to the point I have to pause replications for 24h just so it can somewhat catch up.  

28-30 November
Madrid, Spain
Discover 2017 Madrid
Join us for Hewlett Packard Enterprise Discover 2017 Madrid, taking place 28-30 November at the Feria de Madrid Convention Center
Read more
HPE at Worldwide IT Conferences and Events -  2017
Learn about IT conferences and events  where Hewlett Packard Enterprise has a presence
Read more
View all