LVM Snapshots and BBWC

[wobbe]

I'm making LVM snapshots of running ESXi4 VM's stored on an NFS server. (Debian 5)
I'm a bit worried about the BBWC.
It is set for 75% (96MB) write cache, so now I'm wondering what will happen to all those outstanding write actions when I make a snapshot.
Will every thing be synced to disk when I use lvmcreate or do I need to perform other actions?

[Matti_Kurkela]

The Linux storage system is a multi-layered thing, like the OSI model in networking.

When a write operation has been successfully stored into BBWC at the hardware layer, it's completely equivalent to "stored to disk" for all the upper layers, including LVM.

The creation of a snapshot is a LVM-level action. To do it, LVM reads some blocks, writes some blocks and presents the result as a snapshot available for use. When the LVM does it, it cannot label those read & write operations in any special way. For the lower layers, the reads & writes that are done because of a snapshot operation are just like any other reads & writes.

-----

By your question, I think you may have misunderstood the nature of the LVM snapshot. It *looks* like a complete copy of your LV as it existed at snapshot creation time. It even *acts* like a copy: you can write to it, if you wish. But in reality, it's anything but a copy.

The Linux LVM snapshot is an implementation of Copy-on-Write technique to disk management.

http://en.wikipedia.org/wiki/Copy-on-write

After a snapshot is created, the snapshot mechanism intercepts all write operations to the original LV. If the block has not yet been written since the snapshot was created, the original data of that block is copied to the snapshot before the write is allowed to happen. This happens on the first write of the block only: if same the block is written to a second time, the write will proceed as usual - the state of that block at snapshot creation time is already stored in the snapshot LV.

When you're reading a snapshot, if the data on the original disk block is still the same as at the time the snapshot was created, the block is read from the _original_ LV. If the block has been changed, the snapshot mechanism will find a copy of the old state of the block from the snapshot LV.

This has the advantage of making both snapshot creation & removal operations very fast. The disadvantage is, if your original LV is lost, you discover that your snapshot is most likely full of holes too (=data that has not been copied because it has not changed).

As a RedHat instructor put it: "A LVM snapshot *is not* a copy of your data. It is a clever tool that allows you to *make* a copy of your data, with a minimal impact to your production."

MK

[wobbe]

Thanks MK for the extensive LVM COW write up. I do not intend to keep the snapshot, I rsync the data to another server and after that remove the snapshot a.s.ap. I'm trying to find out if this is a good idea when there could be 96MB data in the contollers cache. Since I'm snapping running VM's I have serious doubts about my procedure.
altough it seems! to work just fine?)

[Matti_Kurkela · Accepted solution]

You know that BBWC does not lose data with normal reads and writes. At the hardware layer, the operations that are done to build the snapshot are just regular reads and writes, no different from any other reads and writes.

When the LVM copies a block from the SmartArray original to snapshot, the only way it can get the block is to request it from the SmartArray controller. The controller alone knows where the data is: if the up-to-date data happens to be in the BBWC, that's where the data is retrieved from. For the LVM, there is no difference whether the snapshot source data is in BBWC or on disk.

If your snapshot LV is on a SmartArray controller, the snapshot data is written first to the BBWC (again), and to the disk when the SmartArray decides it's a good time to do so. Again, when you read the snapshot there is no difference whether the data is in BBWC or on disk.

In short, the presence or absence of BBWC has *no special effect* to snapshot operations. If there is a BBWC, it just means the SmartArray controller has a better ability to optimize write operations, so you get better performance in all write operations.

MK

[wobbe]

Off cource, stupid me. I don't know what I was thinking. Thanks for setting me strait.
I guess I just want to find issues with my setup becasue I won't trust it. :-)