NVMe and I/O Topologies for Dell EMC Intel and AMD PowerEdge Servers

If you are a server user considering Non-Volatile Memory Express (NVMe) storage for your infrastructure, then you are seeking to invest in top-of-the-line performance. Leveraging a PCIe interface improves the data delivery path and simplifies software stacks, resulting in a significant latency reduction and bandwidth increase for your storage data transfer transactions.

PowerEdge rack servers have unique configurations that are designed for specific value propositions, such as bandwidth, capacity or I/O availability. At times it can be a challenge to determine which configuration is best suited for your intended purpose!

Also Read: DEA-1TT4: Dell EMC Information Storage Management (DECA-ISM)

We at Dell EMC would like to simplify this process by providing the value propositions for each of our PowerEdge rack configurations; to help our customers choose the right configuration for their objectives. With this, we have also provided detailed illustrations of NVMe and system I/O topologies, so that customers can easily route and connect their best hardware configurations, and optimally design and configure customer software solutions and workloads.

We can first look at one of our Intel-based rack servers, the R740xd. There are two suggested NVMe and I/O configurations that have unique value propositions:

PowerEdge R740xd with x12 NVMe drives (Maximized Bandwidth)

Figure 1: PowerEdge R740xd CPU mapping with twelve NVMe drives and twelve SAS drives

This 2U R740xd configuration supports twenty-four NVMe drives. The NVMe drives are connected through PCIe switches, which allows the system to overprovision PCIe lanes to more NVMe drives while persevering I/O slots, therefore enabling low latency CPU access to twelve devices per CPU. Performance can easily be scaled for various dense workloads, such as big data analytics. This configuration appeals to customers wanting to consolidate Storage media to NVMe (from SAS/SATA). Customers requiring large capacity with the low latency of NVMe will benefit from this configuration, with up to 24 NVMe drives available for population.

PowerEdge R740xd with x24 NVMe drives (Maximized Capacity)

Figure 2: PowerEdge R740xd CPU mapping with twenty-four NVMe drives

This 2U R740xd configuration supports twenty-four NVMe drives. The NVMe drives are connected through PCIe switches, which allows the system to overprovision PCIe lanes to more NVMe drives while persevering I/O slots, therefore enabling low latency CPU access to twelve devices per CPU. Performance can easily be scaled for various dense workloads, such as big data analytics. This configuration appeals to customers wanting to consolidate Storage media to NVMe (from SAS/SATA). Customers requiring large capacity with the low latency of NVMe will benefit from this configuration, with up to 24 NVMe drives available for population.

Next, we can look at one of our AMD-based rack servers, the R7425. There are two suggested NVMe and I/O configurations that have unique value propositions:

PowerEdge R7425 with x12 NVMe drives (Maximized Bandwidth)

Figure 3: PowerEdge R7425 CPU mapping with twelve NVMe drives and twelve SAS drives

This 2U PowerEdge R7425 configuration supports twelve NVMe drives and twelve SATA/SAS drives. Eight of the NVMe drives are connected directly to the CPU and four of the NVMe drives are connected to CPU1 through a PCIe extender card in I/O slot 3. Customers supporting workloads that demand maximum NVMe and storage performance will need maximum bandwidth to drive the best throughput (GB/s) performance connect to the devices.

PowerEdge R7425 with x24 NVMe drives (Maximized Capacity)

Figure 4: PowerEdge R7425 CPU mapping with twenty-four NVMe drives

This 2U PowerEdge R7425 configuration supports twenty-four NVMe drives. Two PCIe switches are included, which allows the system to overprovision PCIe lanes to more NVMe drives while persevering I/O slots, which are then connected directly to the CPU. This configuration maximizes NVMe capacity and reserves slot 3 for additional I/O functionality but has a lower overall bandwidth. This configuration appeals to customers wanting to consolidate storage media to NVMe from SAS/SATA. Customers requiring large capacity with the low latency of NVMe will benefit from this configuration, with up to 24 NVMe drives available for population.

Each PowerEdge server sub-group has a unique interconnect topology with various NVMe configurations to consider for implementation. To achieve your data center goals with your NVMe investments, it is critical to understand your NVMe topology, as well as why it is the best option from a value prop point of view.

For the full list both Intel-based and AMD-based PowerEdge rack server NVMe and I/O topology illustrations, as well as explanations for each configurations value prop, please view the full NVMe and I/O Topologies Whitepaper now.