Alliances
1819803 Members
2926 Online
109607 Solutions
New Article ๎ฅ‚
Patrick_Lownds

Azure Arc enabled data services

I wrote my first blog about Azure Arc enabled servers back in early 2021. Wow, things with Azure Arc have changed since then! The one thing that has not changed is the question why use Azure Arc? Customers are increasingly requiring an HPE_ELEMENT_Blog.jpganswer to managing and governing their progressively complex environments.

Often enterprise customers have a variety of application architectures and patterns. Alongside this, these same customers have a diverse infrastructure that span differing locations e.g. from the data centre to branch offices and the edge.

Those same enterprise customers are frequently integrating public cloud properties like Azure, AWS and Google into their environments. This all presents a complex problem, especially when it comes to areas like management, security, compliance, and governance.

The big Azure Arc picture

Microsoft has two categories for leveraging Azure Arc. The first category is Azure Arc enabled infrastructure and the second category is Azure Arc enabled services. With Azure Arc enabled infrastructure you connect and operate hybrid resources as native Azure resources. With Azure Arc enabled services, you deploy and run Azure services outside of Azure, but you still operate those services from within the Azure control plane.

Fig1 - Azure Arc Single Control PlaneFig1 - Azure Arc Single Control Plane

Azure Arc enabled infrastructure is considered an IaaS solution. Customers have virtual machines with differing specifications running on differing hypervisor platforms and you can Arc enable them to operate those virtual machines as you would any native Azure resources. Here you can apply Azure security, policy, and governance to these virtual machine instances. There is an Arc agent that sits in the virtual machine and this helps with connectivity to Azure. See the following URL for more insight into the Azure Arc Agent and what else the agent does https://docs.microsoft.com/en-us/azure/azure-arc/servers/agent-overview

Connectivity is via a Public Endpoint (over the Internet), Proxy Server or Private Endpoint (VPN or ExpressRoute). See the following URL for more insight into using Private Endpoints, which is in preview currently https://docs.microsoft.com/en-us/azure/azure-arc/servers/private-link-security

The first offering is Azure Arc enabled servers and you can read more about Azure Arc for servers and getting started with Azure Arc enabled servers here at my previous blog post https://community.hpe.com/t5/Alliances/Getting-started-with-Azure-Arc-for-Servers/ba-p/7119099#.YoOlhVTMKUk 

The second offering is Azure Arc enabled SQL server, which is different from Azure Arc enabled data services, effectively this is a SQL Server instance running inside a virtual machine (IaaS).

The third offering, which is in public preview is Azure Arc enabled Kubernetes. If you have a Kubernetes cluster on-premises e.g. running on Azure Stack HCI, VMware vSphere, or in a non-Microsoft cloud property e.g. AWS or GCP, you can manage and operate those Kubernetes clusters natively from Azure.

The fourth offering, which is in public preview is Azure Arc enabled VMware vSphere. If you have VMware vSphere servers, version 6.7 on-premises, you can manage and operate your VMware vSphere environment natively from within Azure.

Fig2 - Azure Arc Enabled InfrastructureFig2 - Azure Arc Enabled Infrastructure

Azure Arc enabled VMware vSphere doesnโ€™t yet support or has not inherited the full set of cloud operations that Azure Arc enabled servers does today e.g. there is no support for Microsoft Defender for Cloud. That said, here is an overview of exactly what Azure Arc enabled VMware vSphere does support:

  1. The capability to install the Azure Arc Agent remotely, leveraging VMware Tools via ARM (Azure Resource Manager) template, Azure CLI, or APIs.
  2. The capability to carry out various virtual machine operations directly from Azure, such as provision, resize, start/stop and delete.
  3. Provides support for self-service provisioning using Azure RBAC (role-based access control).
  4. Provides a single pane of glass view of your infrastructure, with the ability to browse your VMware vSphere resources, such as virtual machines, templates, networks, and storage.
  5. Govern and monitor operations across VMware vSphere from within Azure, with support for Guest Policy Configuration, and via the Microsoft Management Agent (MMA), the collection of logs, inventory, and change tracking, all of which can be collected and stored in a Log Analytics workspace.

There is however a limitation of only being able to connect to a VMware vCenter server with less than 2500 virtual machines. Likely this will change after the public preview.

Azure Arc enabled data services

In contrast to Azure Arc enabled infrastructure, Azure Arc enabled data services are considered a PaaS-based solution. Here customers can leverage their choice of fabric-based infrastructure and bring any supported version of Kubernetes (Azure Arc enabled Kubernetes works with any Cloud Native Computing Foundation or CNCF certified Kubernetes cluster) e.g. AKS on Azure Stack HCI, VMware Tanzu, Amazon EKS, OpenShift (both Azure RedHat OpenShift and OpenShift Container Platform) and Google Kubernetes Engine, to name a few supported Kubernetes clusters and all this allows you to run Azure Arc enabled data services e.g. Azure SQL Managed Instance or Azure Database for PostgreSQL Hyperscale, the latter still in preview currently, on-premises.

Fig3 - Azure Arc Enabled Data ServicesFig3 - Azure Arc Enabled Data Services

Azure Arc enabled data services architecture has also been designed to be open to future data services e.g. CosmosDB, MySQL, and Synapse.

Finally, just to complete the picture, Azure Arc enabled application services, which are considered a PaaS based solution allows you to run Azure App Service, Azure Functions, Azure Logic Apps, Azure Event Grid, and Azure API Management, all of which are in preview, on-premises. Along with Azure Arc enabled machine learning, which is also considered a PaaS-based solution that allows you to build, train, validate and deploy machine learning models on-premises. However, there is flexibility here with machine learning, as customers can train models in public Azure and deploy these models on-premises or vice versa based on data gravity needs.

Connectivity modes

There are two types of connectivity modes for Azure Arc enabled data services. Direct and indirect connectivity modes. With direct connectivity mode, these services are available from Azure all the time (benefits of security, policy, and governance) using standard ports and protocols such as HTTPS/443 and in indirect connectivity mode, you can think of this as a disconnected mode.

Here you operate any management services locally within your environment and only a minimal amount of data must be sent to Azure for inventory and billing purposes. This data needs to be exported to a file and uploaded to Azure at least once per month. You can read more about connectivity modes and the requirements for connectivity at the following URL https://docs.microsoft.com/en-us/azure/azure-arc/data/connectivity.

Azure Arc data controller

Once you have deployed your fabric-based infrastructure and supported Kubernetes platform, you will then deploy an Azure Arc data controller. The Azure Arc data controller is a series of Kubernetes pods that provide the coordination and orchestration of Azure Arc data services using Kubernetes APIs and any CustomResourceDefination (CRD) APIs.

The Azure Arc data controller can be deployed in direct connectivity mode using the Azure portal. To deploy with direct connectivity mode, you first need to connect to your Kubernetes cluster using Azure Arc-enabled Kubernetes. This connection will deploy Arc agents on your Kubernetes cluster and connect it to Azure. Arc agents are used in direct connectivity mode to coordinate Azure Resource Manager (ARM) requests in the Kubernetes cluster, upload inventory, billing, logs, and metrics collected by the Azure Arc data controller. You can also deploy the Azure Arc data controller in indirect connectivity mode using the Azure CLI (leveraging the arcdata extension), Azure Data Studio, or KUBECTL. Once the data controller is deployed, you can then deploy the services (Azure SQL MI or PostgreSQL).

Fig4 - Azure Arc Data ControllerFig4 - Azure Arc Data Controller

What Azure Arc data services can I deploy?

With Azure Arc enabled data services you can have an always current version of SQL (either Azure SQL MI or PostgreSQL), with elastic scale (limited by the capacity of the fabric-based infrastructure), the ability to deploy an instance in seconds, including automation to both scale up or out as required, and all with a unified management interface that provides a single view for your on-premises, with a cloud consistent set of workflows, giving you consistency across your environments.

Azure Arc enabled SQL Managed Instance is available in two vCore service tiers (General Purpose and Business Critical). The General Purpose service tier is designed for generic workloads and has the feature set of SQL Server Standard Edition, with restriction limits on the amount of CPU and memory (24 cores and 128 GB of RAM respectively). There are a number of other varying capability differences and so it is worth reviewing the following document https://docs.microsoft.com/en-us/azure/azure-arc/data/service-tiers

Fig5 - Deploying Azure Arc enabled SQL Managed InstanceFig5 - Deploying Azure Arc enabled SQL Managed Instance

Business Critical has the feature set of SQL Server Enterprise Edition and has unlimited restrictions on the amount of CPU and memory (only limited by the capacity of the fabric-based infrastructure) plus this service tier has a higher level of high availability (built in Availability Groups).

Azure Arc enabled PostgreSQL Hyperscale comes in three deployment modes (a basic form used for functional validation purposes only, a simple instance that is ready to be scaled out, and a scaled-out form that is ready to meet your application needs).

In the basic form, you have one instance that serves as the coordinator and worker node. The simple instance is instantiated ready to be scaled out, and you define the number of worker nodes and distribute the data. In the scaled-out form there are three or more instances of PostgreSQL Hyperscale. One instance is the coordinator, and the other available instances are the worker instances. Azure Arc enabled PostgreSQL Hyperscale will be available in a range of vCores that are charged at an hourly rate (similar to the service in public Azure), more details will likely be available closer to general availability of this service.

Azure Arc offers a simplified management approach that is consistent with public Azure cloud services. Azure Arc provides visibility of your most important assets, along with support for centralised operations and compliance, across a variety of locations, by extending the Azure control layer to on-premises and other public cloud properties. 

For more information on the many ways we can help you, https://www.hpe.com/uk/en/services/pointnext.html.

Patrick Lownds
Hewlett Packard Enterprise

twitter.com/HPE_TechSvcs 

linkedin.com/showcase/hpe-technology-services/ 

hpe.com/pointnext  

0 Kudos
About the Author

Patrick_Lownds