NVIDIA says full-stack codesign can ease AI memory pressure
NVIDIA says designing GPUs, CPUs, networking, storage and software together improves memory use in AI systems. The company says its approach increased agentic throughput per dollar by up to 30x while HBM capacity per GPU grew by roughly 3%.
NVIDIA says larger models, longer contexts and agentic applications are increasing pressure on GPU HBM, as supply tightens and prices rise in the near term. The company says its full-stack codesign — spanning GPUs, CPUs, networking, storage and software — distributes data across HBM, CPU memory, local SSDs and shared storage to get more useful work from each byte and reduce the impact of the so-called memory wall.
According to NVIDIA, HBM capacity per GPU rose by roughly 3% between the NVIDIA Blackwell Ultra and Vera Rubin platforms, while agentic throughput per dollar increased by up to 30x on SemiAnalysis’ AgentX. The company also says it was the first to bring HBM3e and HBM4 technologies into an accelerated computing stack.
Why it matters · editorial interpretation
NVIDIA’s described approach suggests that handling larger models and agentic applications may depend not only on adding HBM, but on coordinating memory, storage, networking and software. The capacity and cost figures are the company’s claims, while the throughput assessment is attributed to SemiAnalysis.
Sources: NVIDIA