Around the Storage Block
Showing results for 
Search instead for 
Did you mean: 

Data Persistence for Kubernetes and OpenShift and More



Persistent Storage_DockerCon_blog.jpgA sense of excitement is in the air as DockerCon EU 2017 kicks off this week in Copenhagen, Denmark. HPE Storage in particular is excited about this event as we are exhibiting several key innovations, including:

  • Container data persistence for Kubernetes platforms such as OpenShift
  • Tools for automating data migration from legacy environments to Docker Containers
  • Multiplatform data persistence for Windows and Linux containers

If you’re like most organizations, you start your container journey with the relatively straight forward task of containerizing stateless applications such as load balancers, API gateways, etc. that don’t preserve data. Once familiar with containers, you look to containerize traditional applications too.

Our goal is simple: Make persistent storage as easy as deploying containers for traditional and cloud-native workloads!  

Ok, let’s dive right into details.

Data persistence for Kubernetes

Kubernetes is quickly emerging as one of the leading container orchestration tools. Persistent Volume constructs in Kubernetes can be quite daunting but has become easier thanks to active Kubernetes SIGs and community of users driving the change.

Currently, the easiest way to provision persistent volumes is to use the “Dynamic Volume provisioning” feature in Kubernetes (K8s as it is popularly referred as). Instead of storage and cluster admins having to go through a manual provisioning process, volumes can be created and provisioned on-demand using a “provisioner” specified in the “storage class.” You can seamlessly deploy container applications on a storage platform optimized for each workload and use Kubernetes to orchestrate storage management functions.

HPE Storage Class Provisioner for Dynamic Volume Provisioning in Kubernetes: Here’s an example of how easy this is. In the demo below, we are deploying a simple WordPress content management application in RedHat OpenShift environment that uses Storage from both HPE 3PAR and HPE Nimble to store persistent data orchestrated by Kubernetes. We have defined two storage classes: “transactional” which uses “hpe/nimble” storage class provisioner and “unstructured” storage class which uses “hpe/3par” storage class provisioner.

The attributes defined under “parameters” represent storage capabilities and data services exposed by the storage class provisioner to the consumer of the storage class. In this example, you see that the hpe/nimble and hpe/3par provisioners are exposing various capabilities which allows Kubernetes cluster admin to build highly sophisticated templates to applications for policy based storage management.Containers_storage_table.jpg

Therefore, K8s cluster admin can define different storage class objects to represent say “Gold”, “Silver”, and “Bronze” policies each with its own unique storage attributes (parameters). Application developers can consume these storage classes in the Persistent Volume Claims without being aware of storage devices that deliver these services. In short, storage classes deliver necessary abstraction to provide loose coupling between application and storage – a key requirement to satisfy distributed application design paradigm defined in 12 factor design patterns commonly used by cloud nativeapplication framworks.

Note: The storage attributes represented by “parameters” for each provisioner vary depending on the underlying capabilities supported by different storage devices.

Dory, the HPE FlexVolume Driver for Kubernetes with OpenShift

A key component of our architecture is FlexVolume Driver (based on out-of-tree FlexVolume plugin) which leverages our Docker Volume Plug-in for storage provisioning and management. FlexVolume driver translates Kubernetes Persistent Volume (PV) requests to corresponding Docker Volume plug-in requests. By the way, FlexVolume Driver interacts with Docker Volume plug-in through Docker remote APIs directly and not through the container engine (we don’t need Docker Container Engine to be present for FlexVolume Driver to work). Therefore, we work with any container engine and runtime supported by Kubernetes. The best part is FlexVolume Driver is Open sourced. We called it Dory (the one that speaks whale!!). We welcome any vendor or open source project with a Docker Volume Plug-in to integrate with Dory to gain instant integration with Kubernetes.Containers_kubernetes.jpgHere’s the demo of HPE StorageClass Provisioner and FlexVolume Driver in action in Red Hat OpenShift environment

Containerize Legacy Applications: We’ll automate the data migration!

We see many customers trying to migrate virtualized applications to containers who quickly run into a brick wall when it comes to data migration.

  • Copying or replicating data sets from virtual environments into volumes used by containers is time consuming and error-prone.
  • Testing application cutovers is challenging and usually involves a separate copy all together.
  • In Terabyte domain, manually copying datasets could be insurmountable and may involve significant downtime, making it an impractical endeavor.

What if there was a way to relinquish control of the data from the hypervisor and put it under control of the container orchestrator and make it native to the container platform? What if there was a way to test cutovers non-disruptively and iteratively until the process has been perfected?

Well, that’s what we did.

The HPE Nimble Storage Docker Volume plugin allows users to import native Nimble volumes into Docker. To get your data from a foreign storage system onto Nimble, a simple VMware Storage vMotion of a Virtual Disk to a Nimble Virtual Volume (VVol) is step one. Step two, disconnect the VVol from VMware and import it to the container platform. Simple!

The following demo illustrates a simple example of a MySQL database running in a virtual machine with the Virtual Disk on a foreign NFS server. The application data is migrated with VMware Storage vMotion and put in control of the container platform using the HPE Nimble Storage Docker Volume plugin.

Persistent storage for multi-platform containers

Docker, Inc. recently announced support for Windows workers in a Docker Swarm cluster. Windows containers and Windows Server 2016 are becoming more and more relevant. It’s likely that we’ll start seeing more multi-platform applications, clusters, and environments that rely on both Windows and Linux containers. So it’s important to address their stateful needs.

Here’s a demo, showing the unified interface and some of the advanced features that the HPE Nimble Storage Docker Volume plugins provide for multi-platform applications that rely on Docker Volumes, regardless if these are XFS volumes for Linux or NTFS volumes for Windows AND Linux. We deploy a multi-platform application, a Java application running in Linux container and a Microsoft SQL Server running in a Windows container. The application requires persistent storage to store assets the user uploads in the application and the SQL Server which requires persistent storage for its database.

A powerful ability the HPE Nimble Storage Docker Volume plugin provides is to clone Docker Volumes in-place, allowing full application stack cloning, including the data, across multi-platform clusters. Now, THAT is exciting!

Oh, by the way, we also enabled Fiber Channel access to 3PAR via HPE 3PAR Docker Volume Plug-in. You can download the recent plug-ins for 3PAR and Nimble from the Docker Store.

Docker Moby.jpegIf this sounds exciting, then come talk to us at DockerCon in Copenhagen. Michael and I will be at the HPE Booth and look forward to discussing your container journey and how HPE can help accelerate you to success!

Alternatively, you can reach us on Twitter. Go ahead, please share your comments and feedback.

Meet Around the Storage Block bloggers:

Srik V  HPE Storage.jpeg

Srikanth Venkataseshu, Product Management Lead, Cloud and Storage Automation, HPE Storage @srikseshu


Michale Mattsson HPE Nimble Storage.jpg

Michael Mattsson, Technical Marketing Engineer, HPE Storage @drajen



0 Kudos
About the Author


Our team of Hewlett Packard Enterprise storage experts helps you to dive deep into relevant infrastructure topics.

June 18 - 20
Las Vegas, NV
HPE Discover 2019 Las Vegas
Learn about all things Discover 2019 in  Las Vegas, Nevada, June 18-20, 2019
Read more
Read for dates
HPE at 2019 Technology Events
Learn about the technology events where Hewlett Packard Enterprise will have a presence in 2019.
Read more
View all