NVIDIA says full-stack codesign can ease AI memory pressure
🎧 Listen to this story
NVIDIA says larger models, longer contexts and agentic applications are increasing pressure on GPU HBM, as supply tightens and prices rise in the near term. The company says its full-stack codesign — spanning GPUs, CPUs, networking, storage and software — distributes data across HBM, CPU memory, local SSDs and shared storage to get more useful work from each byte and reduce the impact of the so-called memory wall.
According to NVIDIA, HBM capacity per GPU rose by roughly 3% between the NVIDIA Blackwell Ultra and Vera Rubin platforms, while agentic throughput per dollar increased by up to 30x on SemiAnalysis’ AgentX. The company also says it was the first to bring HBM3e and HBM4 technologies into an accelerated computing stack.
Why it matters · editorial interpretation
NVIDIA’s described approach suggests that handling larger models and agentic applications may depend not only on adding HBM, but on coordinating memory, storage, networking and software. The capacity and cost figures are the company’s claims, while the throughput assessment is attributed to SemiAnalysis.
Sources
Hardware