Kioxia has started sampling of its first SSDs featuring an Ethernet interface. The drives are meant to disaggregate modern datacenters by separating computing and storage in a bid to increase storage density, lower total cost of ownership, and lower power consumption. The drives connect to local area networks using a 25GbE interface (or two) simplifying creation of storage subsystems that use only NAND flash.
A number of companies have experimented with Ethernet-connected storage devices for years and Kioxia appears to be one of the first makers to introduce its Ethernet SSDs. To build the platform — which consists of the drives themselves as well as Ethernet Bunch of Flash (EBOF) machines that house them — the company teamed up with Marvell, Foxconn-Ingrasys, and Accton.
“The native Ethernet SSD combined with our switches and controllers offer data centers an EBOF solution that lowers their total cost of ownership, increases performance and reduces power as compared to alternative JBOF solutions,” said Thad Omura, vice president of marketing of the Flash Business Unit at Marvell.
SSDs for datacenter disaggregation
Kioxia’s Ethernet SSDs are based on the Marvell 88SN2400 NVMe-oF SSD converter controllers and they use a single or dual 25GbE RoCEv2 RDMA connections (i.e., they are incompatible with existing TCP/IP network). The drives use the NVMe-oF 1.1 protocol and are compliant with the NVMe 1.4 specification.
When it comes to capacities and performance, Kioxia’s Ethernet SSDs come with 1920 GB, 3840 GB, or 7680 GB of usable NAND and are rated for 670K IOPS random read performance (4KB), which is considerably lower when compared to modern enterprise-grade PCIe SSDs. As for sequential read/write speeds, a single 25GbE interconnection can provide a 3.125GB/s throughput, which is not bad assuming that we are talking about sustained performance, but various EDSFF SSDs with a PCIe 4.0 interface typically offer higher performance.
Kioxia’s Ethernet drives use a 2.5-inch/15mm form-factor and are compatible with a prototype 2U Ethernet Bunch of Flash (EBOF) storage machines that accommodate 24 drives that provide up to a 600 Gb/s storage throughput. EBOFs are not equipped with any CPU, DRAM, or HBA, but feature a Marvell-designed 2.4Tb/s Ethernet switch that connects to the main network or daisy chains additional EBOFs.
Right now, Kioxia’s Ethernet SSDs as well as Accton’s EBOFs are generally a proof-of-concept and are designed primarily for those who want to experiment with NVMe-oF solutions. From performance and storage density points of view, the drives can hardly rival modern PCIe SSDs in new form-factors (e.g., EDSFF), so it is unlikely that there will be many operators of large datacenters who would use this hardware for commercial deployments. Meanwhile, adding capacity using Ethernet SSDs is extremely easy, so once Toshiba comes up with a higher capacity and higher performance Ethernet drives, many exascalers will likely get interested.
“By collaborating with Marvell, Foxconn-Ingrasys and Accton to bring an EBOF system to market, we are enabling the true potential of NVMe over Fabrics,” said Alvaro Toledo, vice president of SSD marketing and product planning at Kioxia America. “This opens up a new world of possibilities for cloud data center operators, software-defined storage providers, and server and storage system OEMs.”