The memory bottleneck in generative inference is forcing manufacturers to rethink chip architecture. At Hot Chips 2026, d-Matrix presented Raptor, an accelerator that abandons the traditional configuration of HBM stacks placed beside the processor in favor of face-to-face stacking: a TSMC N4 logic die positioned directly on top of custom DRAM. This physical proximity allows for a bandwidth exceeding 100 TB/s with only 32 GB of capacity, significantly surpassing the approximately 18 TB/s offered by a 192 GB HBM4 configuration.

Add AlexTech.ai asPreferred Source on Google

SRAM-class bandwidth with reduced power consumption

d-Matrix's approach aims to solve the paradox between capacity and speed. While SRAM offers very high bandwidth (up to 300 TB/s) but with limited capacity and high wattage consumption, HBM guarantees density but faces physical I/O limits and prohibitive energy costs: pushing 100 TB/s via HBM would require approximately 1.92 kW of power. Raptor operates at 0.37 pJ/bit, consuming about one-tenth of HBM's energy and bypassing package "beachfront" constraints. Thermal management is handled through liquid cooling, keeping DRAM under 100°C even with hundreds of watts in the stack.

D-Matrix Aims to Disrupt HBM Pricing with Chiplets, Stacked DRAM, and ... — https://observervoice.com/d-matrix-aims-to-disrupt-hbm-pricing-with-chiplets-stacked-dram-and-innovative-3dimc-accelerator-140928/

The context of the DRAM crisis

The presentation comes at a critical moment for the industry: the global shortage of DRAM memory is pushing prices to new peaks, as highlighted by the doubling in cost of the NVIDIA RTX PRO 6000 Blackwell and ADATA's forecasts of a structural shortage until 2034. In this scenario, alternatives that reduce dependence on high-density HBM become strategic. d-Matrix, already known for its partnership with NVIDIA and the 2025 Corsair SRAM accelerator, consolidates its position as a key player in redefining AI infrastructure.

New 3D-stacked memory tech seeks to dethrone HBM for AI inference — d ... — https://www.tomshardware.com/pc-components/ram/new-3d-stacked-memory-tech-seeks-to-dethrone-hbm-in-ai-inference-d-matrix-claims-3dimc-will-be-10x-faster-and-10x-more-efficient

Technical challenges and scalability

3D stacking introduces significant complexities: heat dissipation through TSVs (Through-Silicon Vias) and the IR drop problem in power delivery are non-trivial engineering hurdles. d-Matrix states that a "1-Hi" stack with power density not exceeding 0.5 W/mm² is manageable with liquid cooling. The capacity of only 32 GB per stack suggests that Raptor will not replace HBM for larger frontier models, but will position itself as an ideal solution for workloads where data access speed prevails over pure capacity, such as KV caching or draft models for speculative decoding.