Networking 101: What is NVMe over Fabric?

NVMe over Fabrics (NVMe-oF) is a young technology that’s growing in popularity. Even at its relatively young age, NVMe-oF is already widely incorporated into many network architectures. Why? NVMe-oF offers a state-of-the-art storage protocol that optimizes today’s SSDs. It can be the tool that bridges the gap between direct-attached storage (DAS) and SANs, enabling organizations to support workloads that require high throughputs and low latencies.

Let’s unpack the details, pros, cons, and other common questions about NVMe-oF. 

What is NVMe?

NVM (non-volatile memory) is a type of memory that holds on to its content when the power fails. NVMe is the function that stores the data and allows you to access it. NVMe is the interface for SSDs. Essentially, NVMe is a type of NUMA (Non Uniform Memory Access) that is optimized, scalable, and feature-rich for use with a broad range of enterprise and client systems. NVMe has the ability to support up to 64K commands per queue, while requiring only 13 commands to deliver optimal performance. 

Read more: Networking 101: What is NVMe?

What is NVMe over Fabric? 

NVMe over Fabrics is also referred to as NVMe-oF and non-volatile memory express over fabrics. It is a relatively new protocol specification designed to connect hosts to storage across a network fabric using the NVMe. NVME-oF supports a range of storage networking options including interface into storage systems, scaling to accommodate large quantities NVMe devices, and extending the distance within a data center so that more spread out subsystems can be accessed.

Pre-NVMe data storage could be broken down into three options:

1. iSCSI: iSCSI is an internet-based storage networking standard for transferring small computer systems interface, aka SCCI, using standard Ethernet cables. Operation speeds are in the ballpark of 1GB/s, although 10GB/s or more can be possible with high bandwidth Ethernet cards.
2. Serial attached SCSI (SAS): SAS is a point-to-point serial protocol for transferring data using SAS cables. This option allows you to connect up to 128 drives via host bus adapters with speeds ranging from 3GB/s, 6GB/s,12GB/s to 22.5GB/s.
3. Fibre Channel Protocol (FCP): FCP is a protocol that uses fiber optic (or copper) cables to transport commands. In this instance, fabrics behave as one big switch, avoiding electromagnetic interference (EMI) issues and offering up other network technologies speeds typically from 1 to 128GB/s.

NVMe-oF is the future of data storage when it comes to performance and bottlenecks. Enter NVMe. The next logical step is to add lossless high-speed data transfer capabilities of FC to NVMe. Welcome NVMe-oF — the evolution of the all-flash array. 

Also read: How Data Centers Must Evolve in the Cloud First Era

Benefits of NVME-oF

The benefits of NVMe-oF include power, performance, ability to scale, and ease of use to your network. NVMe-oF offers: 

• Low latency over your network
• Ability for parallel requests
• Increased and optimized performance, including storage array performance
• Reduction of the length of the OS storage stacks on the server side
• Faster end solution with a move from Serial-Attached SCSI (SAS)/Serial Advanced Technology Attachment (SATA) drives to NVMe SSDs
• High speed
• Ability to scale out thousands of other devices;
• Multipath support of the fabric enables multiple, simultaneous paths between the NVMe host initiator and storage system
• Ability to send and receive commands from many hosts and storage subsystems at one time. 

Drawbacks of NVMe-oF 

There are a few drawbacks and challenges of using NVMe-oF, including: 

• Beware the imposter. Although they may work, there are systems that use a proprietary native protocol for your unique solution. It may be the option that works for you, but it’s just good to be aware that it is not NVMe-oF.
• NVMe-oFis relatively new. Because it’s fairly fresh to the market, many NVMe-oF integrations could be potentially incompatible. 
• NVMe-oF may be an issue with newer technologies. It is basically a faster, more effective iSCSI that has copied outdated architecture and concepts from 40 years ago. This makes it less compatible with current technologies and concepts such as fully automated API control, software defined storage (SDS), hyper-converged infrastructure, and distributed storage (DS).
• No support for end-to-end data integrity can create an issue. End-to-end data integrity checking is important when considering the large volume of data that these systems are expected to process at lightning fast speeds. 

The NVMe-oF Future is Now

NVMe-oF allows for deeper consolidation in the data center, making way for faster network speed and broader bandwidth. It also eliminates silos and allows you  to leverage a single, efficient shared storage infrastructure. 

NVMe-oF also reaches out externally to find more efficient ways to retrieve data and avoid bottlenecks, justifying the current buzz about its place in data management.

Read next: Steps to Building a Zero Trust Network