HPE Nimble Storage Tech Blog
cancel
Showing results for 
Search instead for 
Did you mean: 

Tech Preview: Ephemeral Clones of Persistent Storage for Containers

mmattsson
Containers are by nature ephemeral, transient and stateless. It’s a very comforting fact. If you app gets into a state where you don’t want it to be, simply kill the container and start fresh. This works really well if you don’t need to carry state between restarts or your container doesn’t have to carry any stateful data. If your app needs a tiny portion of data, you might want to ship it with the container image understanding that it won't live beyond the life of the container. If your app needs a terabyte-sized dataset, it’s less practical to ship the data with the container image. For dev/test it’s imperative to quickly spin up environments from various known states to run test cases against. It could be a troublesome dataset your customers got themselves into or simply a fresh revision of your production database. Keeping app and data separate is a well vetted design pattern, let's make sure it stays that way and let's explore how ephemeral clones of persistent storage can accelerate dev/test workflows with the ultimate goal of improving quality of your software deliveries.

With NLT-2.2.1 (and 2.2.1 of the managed plugin) we’re introducing a set of features that makes it trivial to refresh clones from their sources by simply restarting the container. I’m presenting this at Microsoft Ignite with a Hybrid IT flavor using a CI/CD workflow using Visual Studio Team Services.

We also have a demo of that particular workflow running in the HPE booth and it’s available on YouTube for your convenience:

Motivation

While using ephemeral volumes with Docker specifically isn’t anything new, it lacks the ability to specify options and the method seem ill fit for use cases with external volume plugins. An example:
$ docker run --rm -it --volume-driver nimble -v /data alpine
/ # mount | grep /data
/dev/mapper/mpathe on /data type xfs (rw,seclabel,relatime,attr2,inode64,noquota)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

This will create a volume on the fly with a unique ID and attach it to /data. When the container exits, the volume gets removed. While it may seem we tailored these capabilities against this particular limitation, it became more of a nuisance while we built out our CI/CD use case with our Kubernetes FlexVolume driver. The FlexVolume API does not supply a delete/rm/destroy type API call and if you want to remove a volume you need to use whatever means you may have to free up the resources. In the Nimble case it would be fairly simple as you could use docker volume rm to remove the volume but it would be an out-of-band operation from Kubernetes which is not optimal from a workflow perspective.

New options

We’ve designed a set of new options to allow being more specific what the user wants to provision clones from and decide the fate of the volume once removed.

Let’s dissect the new help output from the Clone and Import Clone sections.
$ docker volume create -d nimble -o help
Nimble Storage Docker Volume Driver: Create Help
Create or Clone a Nimble Storage backed Docker Volume or Import an existing Nimble Volume or Clone of a Snapshot into Docker.

Clone options:
  -o cloneOf=X            X is the name of Docker Volume to create a clone of
  -o snapshot=X           X is the name of the snapshot to base the clone on (optional, if missing, a new snapshot is created)
  -o createSnapshot       indicates that a new snapshot of the volume should be taken and used for the clone (optional)
  -o destroyOnRm          indicates that the Nimble volume (including snapshots) backing this volume should be destroyed when this volume is deleted
  -o destroyOnDetach      indicates that the Nimble volume (including snapshots) backing this volume should be destroyed when this volume is unmounted or detached

Import Clone of Snapshot options:
  -o importVolAsClone=X   X is the name of the Nimble Volume and Nimble Snapshot to clone and import
  -o snapshot=X           X is the name of the Nimble Snapshot to clone and import (optional, if missing, will use the most recent snapshot)
  -o createSnapshot       indicates that a new snapshot of the volume should be taken and used for the clone (optional)
  -o pool=X               X is the name of the pool in which the volume to be imported resides (optional)
  -o folder=X             X is the name of the folder in which the volume to be imported resides (optional)
  -o destroyOnRm          indicates that the Nimble volume (including snapshots) backing this volume should be destroyed when this volume is deleted
  -o destroyOnDetach      indicates that the Nimble volume (including snapshots) backing this volume should be destroyed when this volume is unmounted or detached‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Our new keywords are createSnapshot, destroyOnRm and destroyOnDetach. Also, cloneOf now accepts the snapshot parameter which allows the user to specify a snapshot name, either to re-use or in conjunction with createSnapshot create a new snapshot.

We’ve also beefed up the docker volume inspect output to list the snapshots on the volume. Let’s have a look:
$ docker volume inspect mysqlZ
...
        "Status": {
            "Snapshots": [
                {
                    "Name": "testsnap1",
                    "Time": "2017-09-25T21:56:50.000Z"
                }
            ]
...‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

So, this allows the end-user to understand what snapshots are available to use. Let’s give it a try:
$ docker volume create -d nimble -o destroyOnRm myvol1
myvol1
$ docker volume inspect myvol1
...
        "Status": {
          "destroyOnRm": "true"
...‍‍‍‍‍‍‍‍‍‍‍‍‍‍
Deleting mysqlZclone and recreating it would simply revert the volume back to the original state of testsnap1. But let’s flex some of the net new options to demonstrate the convenience of ephemeral clones!

In the following scenario, I’m an imaginary developer wanting fresh data from my production volume for each container restart.
$ docker volume create -d nimble -o destroyOnDetach -o cloneOf=mysqlZ -o snapshot=michaelm -o createSnapshot mysqlZmichaelm
mysqlZmichaelm
$ docker volume inspect mysqlZmichaelm
...
        "Status": {
            "destroyOnDetach": "true",
            "Parent": "mysqlZ.docker",
            "ParentSnapshot": "michaelm",
...‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

We can now see that once this volume will be attached to a container it will be destroyed. A custom snapshot named michaelm was created on the parent volume mysqlZ.docker.

Let’s see how this works:
$ docker run --rm -it -v mysqlZmichaelm:/data alpine df /data
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/mpathg    31441920     32944  31408976   0% /data
$ docker volume inspect mysqlZmichaelm
[]
Error: No such volume: mysqlZmichaelm‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Ephemeral Clones in a nutshell!

To rm or not rm

Not necessarily associated with ephemeral clones but slightly related. Our Docker Volume plugin supports setting parameters for the plugin itself that only the system administrator of the host system can control. We have one particular option as a safeguard for mistakenly removing a volume that wasn’t destined to be destroyed entirely.
In /opt/NimbleStorage/etc/docker-driver.conf:
nos.volume.destroy.when.removed=false‍‍‍‍‍‍‍‍‍
This means that volumes being deleted are not actually deleted from the array itself, they’re simply set offline. This could be a source of frustration as volumes created intentionally for temporary use won’t get destroyed. Setting the above flag to true will certainly destroy the volume when instructed to, but what if we could be a bit smarter about this?
Creating a volume for temporary use, that you don’t necessarily want to use from a clone, you can easily add the destroyOnRm flag to truly destroy the volume upon removal.
$ docker volume create -d nimble -o destroyOnRm myvol1
myvol1
$ docker volume inspect myvol1

        "Status": {
          "destroyOnRm": “true"
…‍‍‍‍‍‍‍‍‍‍‍‍‍‍

This instructs the plugin to forcefully destroy the volume when docker volume rm is issued. There’s no way to recover the volume! Tread with caution.

Summary

There are some really advanced use cases we can accommodate with this functionality, especially in the CI/CD space where you might want to have multiple jobs running in parallel with its own instance of a particular dataset. What also is neat about this functionality is that we enable all our supported container platforms to take advantage of ephemeral clones, Docker, Kubernetes and Mesos!

I hope to meet some of the Microsoft container experts at Microsoft Ignite this week, next month is DockerCon EU. We have a packed booth where I’m representing HPE Storage, stop by for a chat and some awesome demos that will blow you away!
About the Author

mmattsson

Events
See posts for dates
See posts for locations
HPE at 2018 Technology Events
Learn about the technology events where Hewlett Packard Enterprise will have a presence in 2018.
Read more
See posts for dates
Online
HPE Webinars - 2018
Find out about this year's live broadcasts and on-demand webinars.
Read more
View all