Guest Blogger: Maurice De Vidts, SQL Engineer – ESP Database and Big Data Solutions Engineering
Microsoft SQL Server 2019 has introduced a Big Data cluster feature that enhances SQL Server in several ways. It provides a scale out big data processing capability and it also augments data interaction between SQL Server databases, 
Unifying structured and unstructured data yields exponentially more insights that plain unstructured data analytics.
Microsoft SQL Server Big Data Clusters have a data hub that helps bridge traditional structured data silos with unstructured data and enables analytics without performing ETL transformations via data virtualization.
In addition, Microsoft designed the cluster to run and scale on containers. This allows for quick and easy cluster deployments along with a higher and more efficient resource utilization, lowering datacenter operational costs.
The container platform choice ensures that databases can also be reliably and reproducibly deployed on-prem or in the cloud without risk of breaking platform specific configurations. This is key to operating in today’s hybrid environments.
HPE Synergy composable infrastructure and HPE Nimble storage provide an excellent platform on which to build a unifying data lake using Microsoft SQL Server Big Data Clusters. The ability to quickly scale compute, storage and memory also helps provide the clusters with compute, storage, spark and data nodes tailored with the right resources without the need to spend weeks adjusting servers and storage cabling and configurations.
In our SQL Big Data Cluster evaluation, we built two configurations using Ubuntu and Kubernetes:
- Dev/Test environment sized with 70TB of storage
- Medium cluster sized with 140TB of storage
Both configurations can be scaled up further within their rack. In this blog we highlight the basic cluster build.
HPE Synergy advantages
HPE Synergy composability makes setting up and growing the cluster an easy task. Adding compute modules to the infrastructure and applying template profiles got our servers up and configured to our design quickly and without errors. In addition, the Synergy platform provides local, zoned and SAN storage access proving flexibility while meeting the specific storage needs of the cluster.
The Fabric management features of HPE Synergy systems offers the ability to disaggregate fabrics and configure variable bandwidth to each physical node. This allows quick and easy customization and ability to fine tune the network between nodes without the complexity of adding NIC cards or knowing the design up front. The high bandwidth/low latency interconnection between modules reduces cabling and yields network performance where needed.
HPE also provides tight container storage integration with its Volume driver for Kubernetes Flex Volume plugin and upcoming Container Storage Interface (CSI) implementation. We deployed Nimble Volumes using dynamic provisioning and it was very straightforward to configure storage for the Storage nodes in the SQL Server Big Data Cluster.
Future Proofing Container platforms
HPE recognizes the value of container platforms and additionally supports Red Hat Openshift as a container orchestrator for several HPE platforms including HPE Synergy. To that end, HPE has partnered with Redhat and developed Openshift container platform solution reference architectures on Synergy (See references below).
Our Medium cluster Implementation
There are many hardware platforms suitable for container deployments. HPE supports OpenShift across several products including Proliant Rack servers, Synergy, and Simplivity. For this implementation we leverage Synergy due to its data center consolidation role and composability
This system was sized for accommodate in excess of 100TB workloads. It is implemented using seven Synergy SY480 Compute modules, Synergy storage modules and HPE Nimble SAN storage arrays. This can scale further in the two Synergy frames as there is room for additional Synergy compute and storage modules. A rack can also be scaled to 3 frames.
The Synergy storage modules provide local storage for the SQL Server Big Data Cluster storage nodes, while the Nimble provides shared storage for the data nodes in the cluster plus storage for the master SQL Server instance.
Conclusion
Microsoft SQL Server Big Data clusters enhance SQL Server environments with numerous features and scale-out compute and storage capabilities. HPE Synergy along with HPE Nimble storage provides an efficient and flexible platform upon which to deploy a container platform and Microsoft SQL Server Big Data Clusters.
Combining the Microsoft deployment utilities with OneView and Synergy composability results in quick, easy and scalable cluster deployments that reduce your time to market and yield faster data access to growing and disparate data sources.
Scaling compute in the Synergy frame is also possible by composing a profile on a four socket module and can be further enhanced with HPE persistent memory featuring Intel Optane Persistent memory.
Applications that are persistent memory aware such as SQL Server 2019 can leverage persistent memory for additional compute and memory gains.
By building a big data cluster using SQL Server Big Data Clusters, and HPE customers can deploy quickly, with better resource utilization and remain flexible in hybrid platform environments.
Resources
For additional information please see the following links relevant to Microsoft SQL Server 2019, SQL Server Big Data Clusters and HPE solutions supporting SQL Server 2019 container deployments.
Introducing SQL Server 2019 Big Data clusters
Microsoft SQL Server 2019 Big Data clusters, What’s new:
HPE Reference Configuration for Red Hat OpenShift on HPE Synergy and HPE Nimble Storage:
Storage:
EIC_Alliances
HPE Alliance Partners
