Groq, the startup acquired by Nvidia, is reportedly preparing next-generation LPUs that replace SRAM with DRAM stacked vertically in 3D, departing from the SRAM design of current chips. According to a semiconductor industry insider, the company will introduce 3D packaging on its next-generation chips, stacking DRAM instead of SRAM.
The choice is a deliberate trade-off. SRAM is very fast but has limited capacity, while DRAM is slower but allows much larger capacities. With 3D stacking, several DRAM layers are connected vertically through TSVs (through-silicon vias), similar to how HBM is built today. Stacking reduces the occupied area and improves energy efficiency compared with traditional DRAM packages.
Inside the LPU: Deconstructing Groq’s Speed | Groq is fast inference...
Groq's focus therefore seems to shift toward memory capacity, while retaining some of the high bandwidth its SRAM-only chips offered. The reasoning is tied to model growth: the larger models become, the more it matters how much memory a single chip can hold. Some hints also suggest Groq will keep DRAM and SRAM chips separate, leaving the industry to choose the most suitable option.
Nvidia's LPX context
Current Groq LPUs pack 500 MB of SRAM with 150 TB/s of bandwidth; a rack of 256 chips reaches 128 GB of SRAM and 640 TB/s of scale-up bandwidth, with 1.20 PFLOPs FP8 per chip. Nvidia presented its Groq 3 LPX as part of the Vera Rubin ecosystem to improve inference performance, and the LPX rack has entered full production, marking the commercialisation of technology from its largest acquisition. Nvidia says Rubin GPUs and LPUs together can deliver up to 35 times more inference throughput per megawatt for trillion-parameter models. The deal was reported at around $20 billion in December 2025.
Groq's choice fits a broader memory race for AI, where capacity has become the main constraint. As already reported, d-Matrix presented Raptor, an accelerator that stacks logic on DRAM to reach over 100 TB/s with 32 GB, while Sandisk and SK Hynix are developing High-Bandwidth Flash to bring GPU memory to several terabytes. Groq's path runs the other way from SRAM, which is smaller but faster.
No comments yet. Be the first!