NVIDIA Reportedly Plans GPU-Direct Storage for Vera Rubin, Raising Expectations for HBF Beyond HBM

　　As AI models continue to scale, HBM may struggle to meet future memory-capacity demands, prompting industry experts to view GPU-driven storage architectures as a potential next frontier. According to The Elec, NVIDIA and Amazon are reportedly advancing storage architectures that allow GPUs to directly control storage devices such as SSDs. NVIDIA is said to plan the introduction of GPU-Initiated Direct Storage Access (GIDS) starting with its Vera Rubin AI platform, a shift that could accelerate the emergence of high-bandwidth flash (HBF), the report notes.

　　Citing Song Ki-hwan, a professor in the Department of System Semiconductor Engineering at Yonsei University, the report explains that GIDS goes beyond existing GPU Direct Storage (GDS) architecture. Under GDS, CPUs issue data requests to storage devices before data is transferred to GPUs. GIDS advances this by allowing GPUs to access storage directly, bypassing CPUs and DRAM.

　　Both GIDS and GDS aim to overcome data-transfer bottlenecks tied to traditional von Neumann computing architectures. Microsoft and AMD are also said to be exploring similar approaches. The report, citing Song, adds that traditional data-transfer methods are inefficient because CPUs are structurally limited in thread processing, while GPUs can generate tens of thousands of parallel threads. Song also notes that GPU-HBM data transfer already accounts for roughly half of total system power, strengthening the case for HBF architectures that place ultra-fast NAND closer to GPUs to address future AI bottlenecks.

　　GIDS Could Accelerate HBF and Expand NAND’s Role in AI Memory

　　The emergence of GIDS could allow NAND storage to take on a larger role in AI memory systems while easing pressure on HBM capacity. As the report notes, this shift would require higher-performance NAND flash capable of keeping pace with GPU processing speeds. One proposed approach is high-bandwidth flash (HBF), which stacks NAND flash vertically in a structure similar to HBM and connects it using through-silicon vias (TSVs).

　　The report notes that NAND flash offers roughly 30 times higher bit density than DRAM, enabling far greater memory capacity in a similar footprint. According to Song, combining six HBF units with two HBM units could increase GPU memory capacity more than 16 times, from 192GB to 3,120GB, potentially supporting AI models with parameter sizes around 16 times larger than current architectures.

　　Still, NAND flash has endurance limits, typically supporting only around 100,000 write-and-erase cycles versus DRAM’s near-unlimited write capability. As a result, HBF is seen as better suited for storing AI model parameters, which remain largely unchanged during inference and function as read-only workloads.

　　Meanwhile, memory makers have also been exploring GPU-driven memory architectures. According to an Edaily report last year, sources said Samsung Electronics is actively researching next-generation high-performance Z-NAND. The company is also developing GIDS technology that would allow GPUs to directly access Z-NAND-based storage devices. If implemented, GPUs would be able to access Z-NAND devices without intermediaries, potentially shortening processing times for AI workloads.