storage reclamation and VAAI unmap support on SimpliVity NFS datastores

[HughletClapbrd]

While I see that our SimpliVity NFS 3 datastores are "NAS VAAI Supported" does anyone know if they support the UNMAP feature specifically?

What is the process for reclaiming unused space in a Windows VM's disk? Should it happen automatically via the UNMAP command?

Or do I revert to the old process of "sdelete" to zero out the blocks then "storage vmotion" back and forth between SVT datastores?

I have a Windows Server 2012 R2 that was recently migrated to our SimpliVity cluster which had a 4TB disk with data that was previously deduped with Windows dedup. We expanded the drive and hydrated the data again so that we could restore file backups that SimpliVity makes (since SimpliVity doesn't support restoring files from WIndows dedup volumes). Ended up with a much larger disk about half used with a lot of unclaimed space, but SimpliVity is showing the disk taking close to its capacity from the datastore.

[kylerj]

The unmap feature wouldn't necessarily work in the SimpliVity world as our used blocks are determined from the Guest OS perspective. When a block is deemed to be no longer needed, defrag and garbage collection is handled by the OVC automatically. However, if the OS is still using that block or that block as not been freed from the Guest OS, the SimpliVity still thinks the block is in use. Therefore, sdelete would be the best option to zero out the blocks.

I'm not particular sure of the storage vmotion step in reclaiming space as it doesn't change much on the SimpliVity side of things besides changing the datastore (which are all shared anyway) and UUID's within the SimpliVity database.

Also, since the datastore and its size is logical, when you state "SimpliVity is showing the disk taking close to its capacity from the datastore" are you referring to the physical capacity as well. Because the size of the datastore can be increased if that logical construct in the bottleneck for your capacity issues. However, if you are running out of space physically, I recommend you open up a support ticket for additional help.

[HughletClapbrd]

Thanks for your response. Yes, I echo your concern about the sdelete/storage vmotion method because I know with VMFS, null blocks are not reclaimed if the two datastores have the same block size. However on NFS datastores, things become very vendor specific, and like you say, SimpliVity may not see a cause to migrate the data at all.

In terms of the disk on hand, the Windows VM reports 5.51 TB of used space on a 9 TB volume (we already shrunk it via disk management from 11 TB). 

Using the following command:

dsv-balance-show –showHiveName –consumption –showNodeIP

An OVC reports the VM in question using 9.33 TB. The VMware datastore is reporting the same (as this VM is the only one in it currently and few backups have been made). Should I use a different command to see the physical capacity used by this VM?

[kylerj]

You won't be able to see the physical size used on a per VM basis unfortunately because of the dedupe and compression algorithms. However, on a node to node basis, you can see how much physical space is utilized in the dsv-balance-show output within the "estimated space remaining" column. Also, within the SimpliVity Plugin home page, you can view the physical capacity remaining for the selected cluster as well.

[HughletClapbrd]

Yes, using dsv-balance-manual, I can see on which nodes the primary and secondary copies are kept for the VM in question. There are very few (2) other VMs of totaling together less than 70 GB in size on these same nodes.

I can see that roughly 50% of the space is used on these nodes with the large VM on it with 7.15 TB remaining from the "Estimated space remaining" column of the dsv-balance-show.

This would seem to indicate there is close to 2 TB of unclaimed space in the SVT store, correct?

[kylerj]

I'm not particularly sure of the total physical size of your node purchased, however, but the total size of your node minus the estimated space remaining (7.15TB) would be how much physical space is consumed on that node from the VMs housed on that node.

For ex)

VM1 = 5TB
VM2=100GB
VM3=500GB

=5.6TB Total Logical Size

dsv-balance-show estimated remaining (Physical)= 10TB

Total Physical Size of Node = 12TB

Because of SVT's dedupe and compression 5.6TB worth of VM data logically equals 2TBs of phyiscal space consumed (12TB-10TB =2TB).

Hope this clarifies things.

[HughletClapbrd]

It clarifies that we have a major problem with storage efficiency on SimpliVity, and it's not just related to unused block reclamation (which must not be supported),  but lousy compression and dedup rates as well.

These nodes are DL380 g10-L42P-22C, so they started with roughly 14.4 TB of physical space. We've got 7.15 TB remaining on the two nodes that have the primary and secondary copies of this VM. As stated previously, these same nodes contain only one other copy of a small VM, so the used space here is mostly that of the new large VM in question.

This large VM is taking at least 7.2 TB or more on SimpliVity. Compared to its original size of 3.21 TB when this data was on a 3PAR RDM and deduped by Windows - we're really going backwards here.

I've created a support ticket a day ago.

[HughletClapbrd · Accepted solution]

Support confirmed that VAAI UNMAP is NOT supported on SimpliVity NFS datastores, which explains the behavior here, and indicates Vmware HCL for SimpliVity is accurate as well.

Because this feature is not supported, if you delete large amounts of data from a disk volume in a VM, you'll need to run sdelete on that volume to reclaim the storage in SimpliVity and delete any backups of that VM (as the space savings will be trapped there until removed or aged-out).

In my case we got back almost 3.5 TB of physical storage capacity on the SimpliVity nodes containing this VM. It is difficult to determine now exactly the dedup and compression rates, close to what Windows gave us, but we're not using 8KB blocks on this volume, so there's that.

[vene]

Silly question : where has the "sdelete"to be executed ? from the VM itself ? from VMWare ?

Would you mind giving an example of the sdelete command you ran ?

Many thanks in advance 

[HughletClapbrd]

sdelete is run on a large disk/volume in a Windows virtual machine where you have deleted a large portion of data, say it's F:. Assuming you download a recent version, the command is:

sdelete -z F: 

Note that the F: volume will appear to fill up as sdelete works, almost to 100% right before it finishes.