Why are Ethernet Storage Fabrics needed, and why are HPE M-series switches an excellent choice for storage connectivity? Simply put, the HPE M-series Ethernet switch family is designed and optimized for high performance, flexibility and value for all server and storage connectivity requirements in you data center.
- Almost all interesting storage was primary (Tier 1) block storage.
- Storage arrays and application servers (usually Unix) were expensive.
- Ethernet networks and customers were not sophisticated about traffic management or network security, so the best way to ensure a secure and performant storage network was to build a physically separate one dedicated only to block storage.
- Fibre Channel was faster than Ethernet: first 1Gb for FC vs. 100Mb for Ethernet, then 4/8Gb FC vs. 1Gb Ethernet.
- FC was lossless and Ethernet lossy.
Customers didn’t mind paying high prices for FC networking because the cost was relatively low compared to the expensive servers and storage arrays. Applications were mostly centralized and scale-up, running on just a few servers, so storage networks weren’t very large. Scale-out storage, open-source software, distributed applications like Hadoop, cloud, and hyperconverged infrastructure were rare or non-existent. Nearly all storage traffic was “north-south” between servers and storage, meaning storage networks supported relatively few ports.
Everything changes
Then came the 20-teens and the IT world was upended by multiple revolutions. Flash made storage faster, exceeding what 8Gb/10Gb connections could sustain. Scale-out storage and distributed applications required many lower-cost nodes (servers) with lots of “east-west” traffic between them, instead of only traffic to a few clients. Open-source lowered the price of applications, X86 servers (running Linux or Windows) lowered the cost of compute, and competition from new storage offerings lowered the cost of storage server hardware.
Cloud drove the use of VMs – and now containers – which required storage networks to deliver more bandwidth, and also to be smarter and more flexible. This enabled VMs and containers to be created, migrated, or replicated quickly, without relying on an expensive and inflexible fabric. In parallel, hyperconverged infrastructure (HCI)—with vendors like SimpliVity (now part of HPE)—has grown in popularity in the enterprise, promoting an architecture that is scale-out, flexible and always built on Ethernet.
In parallel, customers found more ways to store data, leading to an explosion of file and object storage, as well as distributed local storage for applications like Hadoop. As data continued to grow more rapidly than IT budgets, customers began to tier data based on performance requirements and access patterns. For example, a mission-critical database might run on a branded all-flash array, while less critical data might live on an iSCSI-connected hybrid array, or in a hyperconverged cluster. Older files, plus photos and videos, can be put on NAS or object storage.
The increase in storage types and tiers makes a huge difference in storage planning, because all the file and object storage solutions must use Ethernet and can no longer use Fibre Channel. Much of the secondary storage can in theory go on Fibre Channel SANs as long as it works on block storage, but IT budgets cannot afford FC prices for the rapidly expanding secondary storage capacity.
Ethernet becomes the new storage networking choice
All the while, Ethernet has made large leaps in both speed and sophistication, becoming faster than FC. In 2011, 40GbE became available at lower prices than 8/16Gb FC, then in 2015 100GbE started shipping, two years before 32Gb FC. Today 100GbE costs one-third what 32Gb FC costs per port while delivering 3x more bandwidth and significantly lower latency.
Ethernet supports any kind of storage, like block, file, object, or hyperconverged, and any storage tier which includes primary, secondary, archive, and backup. It can be deployed as either a separate storage network or one converged network supporting both compute and storage, which is exactly what many cloud – and all hyperconverged customers do. Plus, modern Ethernet networks support lossless networking, partitioning, encryption, and QoS, along with sophisticated monitoring and management.
The importance of predictably fast performance
It turns out that storage customers want networks that are not just occasionally fast, or even fast on average, but consistently and predictably fast. They want low average latency, but more often set upper limits for 99.9%, 99.99%, or even 99.999% “long-tail” latency. This type of latency measurement means no more than 1-in-1000, 1-in-10,000, or 1-in-100,000 packets, respectively, can have latency above the limit.
This means latency on the switch should be consistently low no matter what packet size, speed combinations, or port combinations are used. Switch bandwidth should be fairly allocated across all ports, no matter which particular ports have been selected. It also should not have any avoidable packet loss due to traffic microbursts. General networking administrators may not worry about occasional packet loss because most higher-level protocols like TCP/IP will just re-transmit lost data, but this can severely impact performance for more critical storage workloads.
In short, when it comes to storage networks, high latency, inconsistent performance, and lost packets are really BAD, while switches that offer consistently low latency, eliminate avoidable packet loss, and provide fair performance across ports are really GOOD. The HPE M-series switch are in the GOOD category for storage.
HPE M-series Ethernet Switch family: Ideal for NVMe over Fabrics deployments
HPE’s M-series Ethernet switches are the first storage-optimized Ethernet switch capable of providing superior RoCE support and for building an Ethernet Storage Fabric (ESF). It offers the highest levels of performance measured in both bandwidth and latency, supporting from 8 to 64 ports per switch with speeds from 1GbE to 100GbE per port. All the HPE M-series models are non-blocking and allow enough uplink ports to build a fully non-blocking fabric. The latency is not only the lowest (300 nanoseconds port-to-port) of any generally available Ethernet switch, but the silicon, software, and buffers are designed to keep latency consistently low across any mix of port speeds, port combinations, and packet sizes. This prevents congestion and helps optimize RoCE performance. The HPE M-series intelligent buffer design ensures physical memory is allocated to the ports that need it most. This assures I/O is treated fairly across all switch port combinations and packet sizes. Other switch designs typically segregate their buffer space into port groups, which makes them up to 4X more likely to overflow and lose packets during a traffic microburst. This buffer segregation can also lead to an uneven performance where different ports exhibit wildly different performance under load – despite being rated for the same speed.
HPE delivers what the market wants: Ethernet Storage Fabric
Fortunately for our customers with growing Ethernet-based storage, HPE offers Ethernet switches specifically designed and optimized for storage networking. The M-series deliver the highest levels of performance at all the relevant speeds, including 1, 10, 25, 40, 50, and 100Gb/s and soon 200/400 Gb/s. They have the lowest latency of any mainstream switch, feature zero packet loss – meaning no avoidable packet loss, for example – due to traffic microbursts, and deliver that performance fairly and consistently across any packet size, mix of port speeds, or combination of ports.
The HPE M-series provides a high-performance Ethernet Storage Fabric solution to support block, file, and object storage, thereby allowing you to build a converged network capable of simultaneously handling compute and storage traffic from iSCSI storage, to traditional arrays, to the newest NVMe over Fabrics arrays. It has unique storage-aware features such as Data Center Bridging (DCB) which includes Enhanced Transmission Specification (ETS) and Priority Flow Control (PFC). It supports iSCSI TLVs, allowing iSCSI traffic to be specifically monitored and managed.
The HPE switches are upgradable in speeds up to 100Gb/s per port, port count and software features, enabling a storage network ready for expansion, faster storage, faster servers, and expanded storage functionality. This combination of features, performance, flexibility and storage-specific features make HPE M-series the perfect solution for both small and large storage networks. And here’s good news: You can order these switches from HPE TODAY.
Bottom line: you can leverage Ethernet storage benefits to deliver higher data throughput, boost efficiency and reduce total cost of ownership. HPE M-series Ethernet switch family offers industry-leading, high-bandwidth and the lowest-latency for a cost effective solution to connect Primary, Secondary, Hyperconverged, Software Defined Infrastructure, NAS, or Object storage systems.
With cost-effective options such as pay-as-you-grow licensing, seamless speed transition from 1GbE to 100GbE and soon up to 400GbE along with support for next-generation storage such as software-defined, scale-out, distributed and NVMe, ensure your M-series fabrics allows for implementing significant speed and architecture upgrades over time.
For more information, please check out the technical white paper “Ethernet Storage Fabrics - Enabling high-performance Ethernet storage connectivity”.
Find Part 1 of this series, “Better, faster and smarter data center fabrics for high performance workloads” here. And – Look for the third part of this series in the coming months. The third installment will focus on high performance networking fabrics for AI workloads, running on NVidia GPUs based servers.
Storage Experts
Hewlett Packard Enterprise
twitter.com/HPE_Storage
linkedin.com/showcase/hpestorage/
hpe.com/storage
StorageExperts
Our team of Hewlett Packard Enterprise storage experts helps you to dive deep into relevant infrastructure topics.
