- Community Home
- >
- Storage
- >
- HPE Nimble Storage
- >
- Array Setup and Networking
- >
- Reclaim space quickly (sdelete alternative)
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-28-2014 07:40 PM
тАО01-28-2014 07:40 PM
Reclaim space quickly (sdelete alternative)
This is related to the previous post outlining why pre-Windows2012 does not return space back to the array. I just can't seem to add a tag for "scriptingblitz"
https://connect.nimblestorage.com/message/1862#1862
There are two main problems with sdelete that I have observed:
1. It does not recognise mountpoints, so you need to add a letter to the mountpoint to run.
2. It is very, very slow.... taking sometimes days to reclaim a couple of TB.
Chris Duck came up with a great idea of using powershell to perform the same kind of task as sdelete, but has the advantage of working with mountpoints - http://blog.whatsupduck.net/2012/03/powershell-alternative-to-sdelete.html
I have pinch this idea and some of his code to create my own version that runs roughly 10x faster. It uses the same method to create a 1GB initial file (of zeros) which takes about 60 sec. From there I just use good old DOS copy to clone the file until there is no more space left of the drive (ie <1GB). The copy commands are surprisingly fast! and on the 10GB connected array it will reclaim at a rate of 200-300MB/sec, which is roughly 1TB/hr. There could be ways to squeeze more performance out of it using other copy utilities that allow multithreads like robocopy but this works well enough for my liking
Basic Usage is for example
powershell .\NimbleFastReclaim.ps1 E:\
powershell .\NimbleFastReclaim.ps1 E:\dir1\mountpointishere\
It will try to log to C:\temp dir but can specify different file. A third parameter can be passed to sleep between copies to throttle the process if desired.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-30-2014 04:40 PM
тАО01-30-2014 04:40 PM
Re: Reclaim space quickly (sdelete alternative)
looks interesting but getting errors that it's not signed and i'm not up on powershell. says execution of scripts disabled. any idea how to make it run to test it without screwing around with signatures? it does not say it's blocked so can't seem to even run it unsigned.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-30-2014 04:46 PM
тАО01-30-2014 04:46 PM
Re: Reclaim space quickly (sdelete alternative)
The default policy for windows is to disable scrips from running. You can set the execution policy to allow it. Run PowerShell as an administrator and type the following command.
Set-ExecutionPolicy bypass
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-30-2014 04:52 PM
тАО01-30-2014 04:52 PM
Re: Reclaim space quickly (sdelete alternative)
Launch powershell prompt and check your settings
Get-ExecutionPolicy
I think the default is RemoteSigned. Change it to Unrestricted first
Set-ExecutionPolicy Unrestricted
Failing that use Bypass instead of Unrestricted. This is the technet reference for Powershell 3.0 but also applies for 2.0
http://technet.microsoft.com/en-us/library/hh849812%28v=wps.620%29.aspx
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-30-2014 04:54 PM
тАО01-30-2014 04:54 PM
Re: Reclaim space quickly (sdelete alternative)
nice - thank you. glad one of us knows P/S!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-31-2014 07:23 AM
тАО01-31-2014 07:23 AM
Re: Reclaim space quickly (sdelete alternative)
Thanks David. So it worked well. Looks like it took several hours on a 2TB volume so haven't been able to confirm a 10x speed increase but it does appear to work.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-01-2014 09:31 AM
тАО02-01-2014 09:31 AM
Re: Reclaim space quickly (sdelete alternative)
Not to hijack this sled, but how are you reclaiming this space once you run the script? My understanding is that running sdelete (or similar) on a volume will fill the available space with zero's, in essence turning a thin provisioned volume into a thick provisioned volume. So if this is one vmdk of many on a vmfs volume, obviously the vmkfstools/unmap command will not apply to the vmdk (only vmfs), that has now expanded in size as a result of the script. So you need to vmotion this vmdk file to a different data store with a different block size, as thin provisioned then in order to reclaim that space and reduce the size of the actual vmdk file, correct?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-02-2014 03:15 AM
тАО02-02-2014 03:15 AM
Re: Reclaim space quickly (sdelete alternative)
Hi C W,
I should have mentioned that I am performing this task on an iSCSI presented volume rather than a VMDK. I have not tried it on a VMDK and I cannot actually test this myself, but the concept might be the same?? When data is written to the disk and then deleted nothing actually changes on disk except the reference to the file. The array therefore stills sees it as used data. Overwriting this with zeros (and of course deleting the file) allows the array to see this as free space again. Be interested to know if the vmfs in between storage and OS makes any difference here...maybe someone has tried it and can comment?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-02-2014 06:51 PM
тАО02-02-2014 06:51 PM
Re: Reclaim space quickly (sdelete alternative)
I have to admit my 10x was a finger in the air estimate based on approx IO rate
I used this on CS460x2 with 10GB network.
Are you on 10GB network? It should take roughly 2.5hrs for 2TB at a guess. Probably wouldn't matter too much on the Nimble Controller as all the 0's should be compressed very efficiently.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-03-2014 03:59 AM
тАО02-03-2014 03:59 AM
Re: Reclaim space quickly (sdelete alternative)
Hi David,
Testing this on a guest connected iSCSI Nimble thin provisioned volume optimised for SQL Logs. Copied 10GB of ISO files. Guest reports 10GB used and Nimble says the same ish.. Deleted the ISO files and ran your script. BINGO! Within 10 minutes the array reported the space was down by 10GB. Genuinely impressed. Nimble could do worse than engage with you.
Kind Regards,
An extremely impressed Nimbler.