Sandisk has reached a critical milestone in the development of its High Bandwidth Flash (HBF) technology, completing the first tapeout of its memory dies. The company now targets 2027 for the shipment of product samples specifically designed for AI inference engines, moving beyond the initial conceptual phase previously discussed on AlexTech.ai.
The HBF architecture aims to solve the memory wall in large-scale AI systems by stacking NAND flash instead of DRAM, mirroring the HBM structure but offering significantly higher density. According to data presented at the "Sandisk Investor Day in Focus 2026" and FMS 2026, the first generation of HBF targets 512GB per stack (composed of sixteen 256Gb dies) with a read bandwidth of 1.6TB/s. This allows for 8 to 16 times the capacity of HBM within a similar power and footprint profile, as NAND is non-volatile and requires no refresh power.
Halving GPU Requirements for Inference
Sandisk's emulation results using the Alibaba Qwen3-480B-A35B model demonstrate a drastic shift in hardware efficiency. In agentic AI workload simulations, a system equipped with 4 HBF-powered GPUs achieved nearly identical output performance (tokens per second) as a system using 8 HBM-powered GPUs. This suggests that by increasing the memory capacity per GPU to 4TB, the number of accelerators required for high-performance inference could be halved.
The efficiency gain stems from the different requirements of AI inference stages. While the "Prefill" stage relies on raw compute power, the "Decode" stage is constrained by memory latency and bandwidth for reading massive KV caches. Sandisk proposes a "Prefill-Decode Disaggregation" (PD Disaggregation) strategy, where GDDR SDRAM handles the prefill and HBF manages the massive KV caches and model weights during decoding.
Standardization and Roadmap
To ensure industry adoption, Sandisk has co-founded the HBF Technology Consortium alongside SK hynix, Google, and Tenstorrent. The development is proceeding under the Open Compute Project (OCP), with Meta recently joining the group. On August 3, a base die specification (version 0.7) was released, defining the interface between the NAND stack and the AI accelerator via UCIe.
The specification outlines three speed grades: G1 (0.384TB/s), G2 (1.536TB/s), and G3 (3.072TB/s). The bus width is set at 1,024-bit, maintaining compatibility with HBM3e standards. While NAND's write endurance makes it unsuitable for training, its high read speed and non-volatility make it an ideal candidate for the static weights of pre-trained models in inference-heavy environments.
No comments yet. Be the first!