Snapshot usage without snapshots

[vladv106]

Hi,

We have a new CS300 on our premises with the latest FW (2.3.9) and we stumbled upon something that we cannot explain.

After migrating one VM on a datastore backed by the mentioned Nimble, we saw that the space occupied reported by the appliance was about 300GB more than what the guest reported. This is normal as the volume was filled in the past and then, afte some file deletion we recovered the space. To reclaim the space on the Nimble volume we ran a script named NimbleFastrReclaim found also on this forum that basically writes 0s until full. No, Nimble is reporting the correct size but the reclaimed space was moved to Snapshot space.

This is what is what we do not understand. Although the volume has no snapshots there is this space occupied.

Can someone help me understand what is really happening?

[vladv106]

OK, just read the Volumes sections from the Admin guide more and there is a mention of hourly hidden snapshots that are taken automatically. This raises one question though: what happens when the volume space is reaching its limit? How can I delete the snapshots?

LE: How can we use the hidden snapshots and how many hidden snapshots are there?

Thanks

[Nick_Dyer · Accepted solution]

Hi Vlad,

The use of hidden snapshots are done to provide additional recovery possibilities from accidental data deletion (it's a feature put into the system from support, due to the amount of calls we get to assist with recovering data!).

Each volume takes a hidden snapshot every hour and are erased when the next is taken. I don't believe there is a function for end users to disable the creation of these hidden snapshots, most likely because the amount of space used for these snapshots is most likely next to nothing - unless you're doing a write-intensive workload such as a Storage vMotion.

[vladv106]

Thank you for the answer.

[jonathan_disley]

Hi Vlad,

As per Nick's answer, there is one hourly hidden snapshot per volume which is overwritten when the next is taken.  You can't get access to these yourself, they are accessible only by Nimble support as a safety net in event that you accidentally delete data and don't have your own snapshot schedule(s) configured.  You therefore need to contact support ASAP if you realize you have deleted data and need it to be recovered from the hidden snapshot.

With regards to what happens when volumes run out of space, this wouldn't really be affected by the hidden snapshots, as unlike some storage platforms where snapshot data consumes the allocated capacity of the parent volume, this isn't the case on Nimble.  If you haven't configured dedicated snap reserve for a particular volume (default behavior) then snapshots simply consume free space from the overall pool.  If you have configured a reserve for a particular volume then the snapshot data will consume space from that reserve.   Either way, the snapshot data doesn't count towards the volume's own % used capacity, it is separate. 

The below screenshot should help with understanding this.  If you look at the middle volume you can see that the configured size is 200GB, and it is currently around 75% full at 146GB used.  However, the total space consumed by the volume and its associated snapshots is 234.6GB (right hand side), larger than the volumes' own configured size.  This is because the 88.57GB of snapshot data is being taken from pool free space, and is not consuming the capacity allocated to the volume (200GB).

 

So the answer to your question is that deleting the hidden snapshot for a volume would do nothing to bring down the volume's capacity usage.  The overall pool of free space from which snapshots consume capacity can be easily managed, and InfoSight alerts you up to 90 days in advance when it predicts that you will reach 90% overall capacity usage on the pool.  At this point you would typically perform housekeeping to ensure you aren't retaining any manually configured snapshots that are no longer required, run a thin provisioning space reclaim in VMware, or start the process to order a new expansion shelf for the array.  Were you to end up critically short of free space it is highly unlikely that deleting the hidden snapshots (which are only 1 hour of changed data per volume) is going to give you back much space.  By heeding the InfoSight alerts and acting in good time this is something you should never have to worry about.

I hope that helps provide a little more clarification.

Regards,

Jonathan

[vladv106]

Thanks Jonathan. Your answer provided more insight.

[jasonwoerner115]

Understanding this is a zombie thread. But in the case where a large volume has a large amount of data deleted from it in a short amount of time, hidden snapshots can consume all available free space on the SAN and force offline any thin provisioned LUNs without space reserve. This, by definition, would happen very quickly and can have catastrophic consequences. It seems completely crazy that potentially very large amounts of space can be consumed on a Nimble array without the owner of the array being able to do anything about it, either to prevent it or mitigate it once it happens. Terrible decision by Nimble.