Reclaim space quickly (sdelete alternative)

[david_tan2]

This is related to the previous post outlining why pre-Windows2012 does not return space back to the array. I just can't seem to add a tag for "scriptingblitz"

https://connect.nimblestorage.com/message/1862#1862

There are two main problems with sdelete that I have observed:

1. It does not recognise mountpoints, so you need to add a letter to the mountpoint to run.

2. It is very, very slow.... taking sometimes days to reclaim a couple of TB.

Chris Duck came up with a great idea of using powershell to perform the same kind of task as sdelete, but has the advantage of working with mountpoints - http://blog.whatsupduck.net/2012/03/powershell-alternative-to-sdelete.html

I have pinch this idea and some of his code to create my own version that runs roughly 10x faster. It uses the same method to create a 1GB initial file (of zeros) which takes about 60 sec. From there I just use good old DOS copy to clone the file until there is no more space left of the drive (ie <1GB). The copy commands are surprisingly fast! and on the 10GB connected array it will reclaim at a rate of 200-300MB/sec, which is roughly 1TB/hr. There could be ways to squeeze more performance out of it using other copy utilities that allow multithreads like robocopy but this works well enough for my liking

Basic Usage is for example

powershell .\NimbleFastReclaim.ps1 E:\

powershell .\NimbleFastReclaim.ps1 E:\dir1\mountpointishere\

It will try to log to C:\temp dir but can specify different file. A third parameter can be passed to sleep between copies to throttle the process if desired.

[Daniel-san]

looks interesting but getting errors that it's not signed and i'm not up on powershell.  says execution of scripts disabled.  any idea how to make it run to test it without screwing around with signatures?  it does not say it's blocked so can't seem to even run it unsigned.

[aherbert23]

The default policy for windows is to disable scrips from running. You can set the execution policy to allow it. Run PowerShell as an administrator and type the following command.

Set-ExecutionPolicy bypass

[david_tan2]

Launch powershell prompt and check your settings

Get-ExecutionPolicy

I think the default is RemoteSigned. Change it to Unrestricted first

Set-ExecutionPolicy Unrestricted

Failing that use Bypass instead of Unrestricted. This is the technet reference for Powershell 3.0 but also applies for 2.0

http://technet.microsoft.com/en-us/library/hh849812%28v=wps.620%29.aspx

[Daniel-san]

nice - thank you.  glad one of us knows P/S!

[Daniel-san]

Thanks David.  So it worked well.  Looks like it took several hours on a 2TB volume so haven't been able to confirm a 10x speed increase but it does appear to work.

[a_user57]

Not to hijack this sled, but how are you reclaiming this space once you run the script? My understanding is that running sdelete (or similar) on a volume will fill the available space with zero's, in essence turning a thin provisioned volume into a thick provisioned volume. So if this is one vmdk of many on a vmfs volume, obviously the vmkfstools/unmap command will not apply to the vmdk (only vmfs), that has now expanded in size as a result of the script. So you need to vmotion this vmdk file to a different data store with a different block size, as thin provisioned then in order to reclaim that space and reduce the size of the actual vmdk file, correct?

[david_tan2]

Hi C W,

I should have mentioned that I am performing this task on an iSCSI presented volume rather than a VMDK. I have not tried it on a VMDK and I cannot actually test this myself, but the concept might be the same?? When data is written to the disk and then deleted nothing actually changes on disk except the reference to the file. The array therefore stills sees it as used data. Overwriting this with zeros (and of course deleting the file) allows the array to see this as free space again. Be interested to know if the vmfs in between storage and OS makes any difference here...maybe someone has tried it and can comment?

[david_tan2]

I have to admit my 10x was a finger in the air estimate based on approx IO rate

I used this on CS460x2 with 10GB network.

Are you on 10GB network? It should take roughly 2.5hrs for 2TB at a guess. Probably wouldn't matter too much on the Nimble Controller as all the 0's should be compressed very efficiently.

[marktheblue45]

Hi David,

              Testing this on a guest connected iSCSI Nimble thin provisioned volume optimised for SQL Logs. Copied 10GB of ISO files. Guest reports 10GB used and Nimble says the same ish.. Deleted the ISO files and ran your script. BINGO!  Within 10 minutes the array reported the space was down by 10GB. Genuinely impressed. Nimble could do worse than engage with you.

Kind Regards,

                       An extremely impressed Nimbler.