Data Center Strategy: Explanation of NVMe over Fabrics (NVMe-oF)
To meet the growing number of data center services and private-cloud applications, IT departments are replacing traditional direct attached storage (DAS) with NVMe flash-based arrays and storage targets that utilize NVMe over Fabrics (NVMe-oF) technologies.
You may be surprised to learn that the NVMe protocol is not limited to connecting flash SSDs by PCIe within a rackmount server or DAS enclosure: it may also be used as a networking protocol. (This is to distinguish it from a network, which may restrict the possible connections possible). When used in this context, NVMe-oF enables any-to-any connections of high-performance, low-latency flash arrays, which obliterate DAS storage with legacy SAS or SATA-based interfaces.
Should your organization unleash the full performance potential of a flash storage investment with the ability to share flash storage across servers? First, let’s examine how utilizing NVMe over Fabrics (NVMe-oF) technologies can help flash arrays or storage targets achieve high-performance and low-latency more efficiently.
Researching flash-based solutions to achieve new levels of performance and efficiency? Here’s what data center administrators need to know to evaluate NVMe flash storage solutions. Questions? Drop us an email at
Introducing NVMe over Fabrics (NVMe-oF)
Previously, the challenge with using NVMe is that other servers cannot benefit from flash storage. Traditionally, the adoption of NVMe by data center users is often to directly connect an NVMe SSD by PCIe within a rackmount server or DAS enclosure. While this improves the performance of a single enclosed system, this greatly underutilizes NVMe and restricts the possible connections, as the flash device is not accessible by other systems beyond the server to which it is attached.
With an NVMe “fabric”, any server can access any NVMe SSD without a dedicated PCIe connection. Simply connect the fabric-based JBOF (Just a Bunch of Flash) expansion enclosure to the network switch, to make it accessible among servers on the network. This gives data center users a more efficient way to use flash storage for cloud and mission-critical applications, with network connectivity options that include:
Ethernet – Instead of landlocking NVMe SSDs into a single-enclosed system, NVMe-oF takes the capabilities of NVMe and shares a flash array over an Ethernet fabric to a collection of servers on a network.
Fibre Channel – For environments already invested in fibre channel (FC) infrastructure, most modern switches and IP switches already have updated firmware to support NVMe-oF network protocol. FC is designed for storage and can support both legacy SCSI traffic and NVMe traffic simultaneously. Since most organizations will integrate NVMe-oF into an existing data center, current support of both protocols enables them to make the conversion at a pace comfortable for them.
Enabled by NVMe over Fabrics (NVMe-oF), IT organizations are able to create a very high performance storage network with latencies that rival direct attached storage (DAS). Now, JBOF DAS can be shared, when needed, among servers.
When to use NVMe-oF?
So, when should you migrate off traditional storage? When IT planners aggressively switch to NVMe-oF is largely a function of need and timing. Data center operators concerned about system performance know that investing in high-performance flash array makes the most sense where the storage target is the current bottleneck.
Flash Array – The first step to NVMe for most organizations is an NVMe Flash Array with traditional networking connections to the storage network. While these systems generate IOPS approaching the millions, the reality is that there are very few workloads that require more than the performance of these systems.
Storage Target – An NVMe-oF storage target can be dynamically shared among workloads – providing an “on-demand” or composable storage resource that provides additional benefits, including flexibility, agility, and greater resource efficiency. By leveraging both NVMe SSDs with NVMe over Fabrics (NVMe-oF), data center operators to create a very high performance storage network with latencies that rival SAS or SATA-based direct attached storage (DAS). Typical workload that benefit from the adoption of an end-to-end architecture are high-read workloads or large block applications.
Next, how do you get all the performance potential out of an all-flash NVMe architecture solution?
Choosing an all-flash NVMe Architecture Solution
Low latency or high transfer rates are of little benefit if they swamp the target application. While these systems generate IOPS approaching the millions, the reality is that there are very few workloads that require more than the performance of these systems. However, there is an emerging class of workloads that can take advantage of all the performance and low latency of an end-to-end NVMe system.
If you’d like to learn more about how to maximize your data center infrastructure by up to 90% while minimizing its footprint, give us a call at (888) 828-7646, email us at or book a time calendar to speak. We’ve helped organizations of all sizes deploy composable solutions for just about every IT budget.
