Tech Preview: GKE On-Prem with HPE ProLiant and HPE Nimble Storage

[MichaelMattsson]

No doubt it was hard miss the announcements (Press releases: HPE, Google) at Google Cloud Next ’19 revolving around Google Anthos. HPE now offers a hyper-converged solution for the GKE On-Prem (Google Kubernetes Engine) component of Google Anthos with HPE Simplivity and a converged solution with HPE ProLiant servers and HPE Nimble Storage. It’s been quite busy after Google Cloud Next, but let's go through the demo we put together for the show floor that amazed customers, prospects and partners alike.Situated at Google Cloud Next '19

Please note that Google Anthos and GKE On-Prem is still in beta. We can expect further exciting announcements when it becomes generally available (GA).

The below diagram illustrates where GKE On-Prem sits in the overall architecture (Source).

Google Anthos OverviewGKE On-Prem is a component of Google Anthos which allows Google customers to run GKE on their own hardware and manage on Google Cloud Platform (GCP). Vendors, such as HPE, partner with Google to create validated designs that allow customers to confidently deploy Kubernetes with end-to-end support in a cloud-native fashion. HPE provides the benefit of choice with both a converged and hyper-converged solution. I can only speculate that HPE’s diverse infrastructure portfolio would hold future solutions and give customers even more granularity in their choices. Google contributes a first-class Kubernetes experience that literally had me blown away.

Solution and Demo
As we walk through the different sections of the demo, let’s have the underlying architecture in mind and what we’re trying to showcase. We guide you through the hoops of deploying a stateful application on GKE On-Prem, integrate it with HPE Cloud Volumes and later clone the replica for use with GKE on GCP.

Overall solution based on HPE ProLiant servers and HPE Nimble Storage:Converged GKE On-Prem solution from HPEThe demo walks through deploying Redis on GKE On-Prem, replicating it to HPE Cloud Volumes and later use the replica volume with GKE on GCP:Workload Overview

The HPE ProLiant and HPE Nimble Storage converged solution provides a traditional three tier stack with compute, network and storage from HPE (hint: we’re flexible on the network part...). VMware vSphere is the IaaS endpoint that GKE On-Prem leverages to provision and manage Kubernetes clusters. Another critical component in the solution is the F5 BIG-IP LTMs (Local Traffic Manager) that provides external connectivity to the Kubernetes clusters that are being provisioned to consumers. Depending on the use cases and expected frontend traffic, the LTMs can be either physical or virtual, running on the same VMware vSphere clusters as the GKE On-Prem clusters.

The demo is split into three main sections, the video is embedded below and discussed separately below.

Provisioning and Cluster Management
The intro gives a bird’s eye view of the virtual infrastructure and how HPE Nimble Storage comes into play by having clusters provisioned on VMware VMFS datastores served by HPE Nimble Storage. If you have a sharp eye we can observe a 10:1 storage reduction out the gate with just three clusters. Each of the clusters are associated with a preconfigured VMware Resource Pool. Not illustrated is the requirement of having a separate partition for each of clusters on the LTMs. In highly dynamic environments where compartmentalizing resources is essential, separate VMware VMFS datastores could be provisioned per user cluster as well for ultimate resource control.

The tool Google provides (gkectl) generates Cluster API objects an administrator would apply to the admin cluster in the solution. I wrote a blog post on my personal blog that double-click on the Cluster API and the immense flexibility it provides to manage Kubernetes clusters. What can be observed in the demo, is an administrator of a user cluster being able to edit the machine deployment and dynamically expand the cluster. The admin cluster will then create a new VM and join the new node to the user cluster, which takes about a minute. Also, the cluster can be contracted in the same fashion but it’s not shown due to time constraints in the demo script. As a sidenote, a brand new user cluster takes about fifteen minutes to provision. Do note that the limiting factor is the bandwidth to the Google Cloud Registry (GCR) combined with the bandwidth from where you run gkectl against the vSphere API. Hosting the GKE On-Prem images in a private registry within the enterprise and an adequately sized VMware vCenter Server along with the gkectl endpoint would cut down deployment times significantly for a true cloud experience.

Increasing the replica count of a machinedeployment

The last part in the first section shows the observability of the GKE On-Prem cluster on the Google Cloud Platform.

Persistent Storage for GKE On-Prem
The solution may take full advantage of the deep integration HPE Nimble Storage provides for Kubernetes. A cluster administrator may define a default StorageClass or custom StorageClass API objects tailored for different workloads and use cases. As we can see in the demo, we leverage HPE Nimble Storage Folders to constrain resources, a Protection Template to enforce Data Protection among a few other unique abilities exclusive the HPE Nimble Kube Storage Controller.

StorageClass API Object using the HPE Nimble Kube Storage Controller

A very trivial Redis workload is deployed on the cluster, further we can observe how we insert a key/value pair and have the Persistent Volume replicated automatically through the Protection Template to HPE Cloud Volumes.

Tech Preview: Integration with HPE Cloud Volumes and GKE on GCP
In the last section we have the data replicated to HPE Cloud Volumes, what can we do with it? What we included in the demo is a visionary element which illustrates the ability to clone a replica volume to use with GKE on GCP. It leverages a StorageClass with the HPE Nimble Kube Storage Controller to allow an end-user to annotate a Persistent Volume Claim (PVC) to instantiate a cloud volume from a replica volume. Once cloned, it can be used for a wide range of purposes, dev/test, CI/CD & ETL pipelines, cloud bursting, reporting and batch processing to name a few.

Summary
The Google Anthos announcement is very exciting in many aspects and it shows that Google is committed to enable the cloud for the traditional enterprise. We’ve seen a lot of talk the last year or so about multicloud. Brace yourselves, it’s finally here! Kubernetes is the only underlying technology that can enable multicloud, not only for your workloads and applications, but also your data. With the announcement came also the ability to run workloads on AWS and Azure, managed by Google Anthos, this ultimately gives the consumer the desired flexibility of porting workloads between private, hybrid and public clouds without compromises. I wrote another personal blog post on the multicloud promises Google Anthos ultimately gives us – enabled by Kubernetes.

Also, Google Anthos Migrate, another beta announced at the same time, will allow customers to lift and transform traditional workloads running in VMware virtual machines to native Kubernetes deployments to ultimately create the fluidity we all envisioned with multicloud. A great blog to better understand Google’s viewpoint from an engineeering perspective is available here.