Cloud-native CI/CD pipelines with HPE Cloud Volumes

[MichaelMattsson]

In an era where it’s expected that everything you touch as a developer or ISV should be cloud-native – it’s not without a tad bit of pride we can showcase a HPE Nimble Storage product using nothing but 100% cloud-native tools and infrastructure.

During Amazon re:Invent and HPE Discover late last year we had put together a show floor demo of a CI/CD pipeline using only SaaS and IaaS available in the public cloud. As illustrated in the embedded diagram, the following services were used:

AWS CodePipeline to build our CI/CD pipeline.
AWS CodeCommit (step 2) for storing source code for our app.
AWS CodeBuild (step 3 and 4) to build and ship our app (in a container).
Amazon Elastic Container Registry (Amazon ECR) for storing our built app.
AWS Lambda (step 5 and 6) to deploy our app in a staging environment.
Runscope API Monitoring (step 7) for testing our app in the staging environment.
Slack for user notifications on build status.
Amazon Elastic Container Service for Kubernetes (Amazon EKS) where our prod and stage app runs.
HPE Cloud Volumes provides persistent storage to our app via Kubernetes.

The “app” is a bare-bone stateless REST API gateway in front of a randomly generated census database (source here). The database in itself is a 400GiB+ MariaDB instance running on EKS with a 2TiB Persistent Volume Claim (PVC). The PVC has been satisfied by the HPE Nimble Kube Storage Controller using the Container Provider for HPE Cloud Volumes. The “production” app runs in a separate namespace where it keeps filling with census data. When user pushes to AWS CodeCommit (step 1) the AWS CodePipeline kicks off. The relevant detail as how it pertains to HPE Cloud Volumes is that for each new build, an Ephemeral Clone gets created from the current state of the database that the app is then tested against. Any tests run on the Ephemeral Clone is discarded whenever the stage app is discarded or a new build comes in, whatever comes first. The production instance of the database is completely unaffected but the cloning operation and the clone itself is IOPS constrained to not consume resources from the parent.

The entire demo project as shown is built upon the example found through AWS Product Manager Chris Barclay as described in his blog post here. The example discussed are available in the aws-samples/aws-codesuite-demo on GitHub. While we made a few updates here and there for our demo that is specific to our circumstances, those update are sitting in a private branch and will be made available upon request.

Meanwhile on YouTube
So, without further ado, lets enjoy Speed DevOps Using Container Storage on HPE Cloud Volumes:

Please leave your feedback in the comments below or on YouTube. Let us know if you need help kickstarting your container projects, cloud-native or not, we got it all covered!