Array Performance and Data Protection
Showing results for 
Search instead for 
Did you mean: 

Nimble sweeping process

Go to solution
Occasional Advisor

Nimble sweeping process

Please can someone explain in detail how the sweeping process works?  I assume it is re-arranging the blocks at the physical layer so that they are contiguous and then updating the logical block pointers to the new locations. Is this correct? If so, does that then produce write amplification on SSDs? Or does the sweeping process interact with SSD garbage collection in some way?   Thank you!


Re: Nimble sweeping process

Data blocks are indexed as they are written. Over time the deletion of snapshots and data leaves stale data blocks. Sweeping removes stale blocks and forms new stripe writes with the remaining active blocks.

Nimble OS employs a lightweight sweeping process. This process is made possible by the indexing of all blocks as they are written. The index continues to track block locations after the deletion of data, snapshots, etc.  This allows the sweeping process to remove the stale blocks and form new full stripe write areas. The result is full RAID stripes of free space so that new data can be written sequentially. This helps maintain consistent performance for both writes and subsequent reads.

write amplification SSDs contain 4 KB blocks that are organized into 16 KB pages. When data is read from an SSD, reads can be completed at the 4 KB granularity level. However, writes to an SSD must be made in 16 KB pages even when writing a 4 KB block of data to the SSD. The read-modify-write approach is used to read a full 16 KB page into memory, change the individual block (or blocks), then rewrite the 16 KB page back to the SSD. Theoriginalpageis markedfor reclamationthroughaprocess knownasgarbagecollection. Writeamplification occurs when significant amounts of read-modify-write operations are triggered, causing data that has not changed to be rewritten to the SSD and increasing write wear on the cells. The visible effects of write amplification for the user are higher latency (operations take more time) and lower endurance (SSDs wear out faster).


How HPE Nimble Storage Arrays Mitigate SSD Limitations HPE Nimble Storage arrays are a compelling portfolio of modern storage systems with extreme flash efficiency. The arrays are designed to maximize the benefits of SSD technology while mitigating its limitations. Mitigating Endurance Limitations HPE Nimble Storage arrays minimize the number of writes and block changes that occur on SSDs by processing changes in memory before sending them to the SSD. For example, the following strategies illustrate ways in which the arrays reduce or avoid read-modify-write operations in the SSD: • The arrays use inline deduplication and compression because postprocess deduplication or compression results in unnecessary read-modify-write operations. • To avoid changes that affect a single block within a page and that lead to read-modify-write operations, writes to SSDs are performed in block sizes (for instance, 512 KB) that align with page sizes. • The TRIM command allows the storage operating system to tell the SSD which of the blocks that have been marked for deletion in the file system have yet to be deleted on the SSD media. Consequently, during garbage collection, the SSD does not attempt to retain blocks that have already been deleted, thus reducing unnecessary read-modify-write operations.

I work for HPE

Accept or Kudo