面对这一困境,我们该如何突破算力瓶颈,构建更高效的AI基础设施?如何在性能、容量、成本和功耗之间找到精妙的平衡点?业界正积极探索并演进出一种多层次的内存与存储体系(Tiered Memory and Storage Hierarchy)。这种架构范式通过在靠近处理器(CPU/GPU)的不同距离上部署不同特性的内存与存储技术,旨在为AI工作负载提供更灵活、更高效的数据访问路径。本文将深入剖析构成这一新兴层次结构中的五项关键技术:高带宽内存(HBM)、本地CPU-DRAM、基于CXL的DRAM资源池、高带宽闪存(HBF)以及GPU-Direct闪存。我们将从时延、带宽、容量、市场成熟度及落地可行性等多个维度进行深度分析,希望能为您的AI基础设施决策提供清晰、可执行的技术洞察。
Managed-Retention Memory: A New Class of Memory for the AI Era - arXiv, accessed on August 8, 2025, https://arxiv.org/html/2501.09605v1
Why the performance of your storage system matters for AI workloads - Micron Technology, accessed on August 8, 2025, https://www.micron.com/about/blog/storage/ssd/why-the-performance-of-your-storage-system-matters-for-ai-workloads
Addressing The Memory Guy's CXL Conundrums - EEJournal, accessed on August 8, 2025, https://www.eejournal.com/article/addressing-the-memory-guys-cxl-conundrums/
Memory Sharing with CXL: Hardware and Software Design Approaches - arXiv, accessed on August 8, 2025, https://arxiv.org/html/2404.03245v1
Optimizing Data Center TCO With CXL And Compression - Semiconductor Engineering, accessed on August 8, 2025, https://semiengineering.com/optimizing-data-center-tco-with-cxl-and-compression/
HBM Memory: Complete Engineering Guide & Design Optimization ..., accessed on August 8, 2025, https://www.wevolver.com/article/hbm-memory-complete-engineering-guide-design-optimization-2025
GDDR6 vs HBM - Different GPU Memory Types | Exxact Blog, accessed on August 8, 2025, https://www.exxactcorp.com/blog/hpc/gddr6-vs-hbm-gpu-memory
NVIDIA GH200 Grace Hopper Superchip Architecture - AMAX Engineering, accessed on August 8, 2025, https://www.amax.com/content/files/2023/12/NVIDIA_Grace_Hopper_Superchip_Architecture_Overview_Whitepaper.pdf
HBM2 vs GDDR6: Engineering Deep Dive into High-Performance Memory Technologies, accessed on August 8, 2025, https://www.wevolver.com/article/hbm2-vs-gddr6
What's the Difference Between GDDR and DDR Memory? | Exxact Blog, accessed on August 8, 2025, https://www.exxactcorp.com/blog/HPC/what-s-the-difference-between-gddr-and-ddr-memory-
DDR4 and DDR5 Performance Comparison, Plus GDDR6 and HBM2 - BittWare, accessed on August 8, 2025, https://www.bittware.com/resources/ddr4-and-ddr5-performance-comparison/
Managing Memory Tiers with CXL in Virtualized ... - SymbioticLab, accessed on August 8, 2025, https://www.microsoft.com/en-us/research/wp-content/uploads/2024/03/2024-FlatMemoryMode-Memstrata-OSDI2024.pdf
NVIDIA Grace Hopper Superchip Architecture In-Depth | NVIDIA Technical Blog, accessed on August 8, 2025, https://developer.nvidia.com/blog/nvidia-grace-hopper-superchip-architecture-in-depth/
CXL Memory - Samsung Semiconductor, accessed on August 8, 2025, https://semiconductor.samsung.com/cxl-memory/
Compute Express Link - Wikipedia, accessed on August 8, 2025, https://en.wikipedia.org/wiki/Compute_Express_Link
The Performance of CXL Memory (Latency and Bandwidth) - My Note, accessed on August 8, 2025, https://0x10.sh/the-performance-of-cxl-memory-latency-bandwidth
Toward CXL-Native Memory Tiering via Device-Side Profiling - arXiv, accessed on August 8, 2025, https://arxiv.org/html/2403.18702v1
OPPORTUNITIES AND CHALLENGES FOR COMPUTE EXPRESS LINK (CXL), accessed on August 8, 2025, https://computeexpresslink.org/wp-content/uploads/2024/11/CR-CXL-101_FINAL.pdf
Sandisk and SK hynix working to standardize High Bandwidth Flash ..., accessed on August 8, 2025, https://blocksandfiles.com/2025/08/07/sandisk-and-sk-hynix-working-to-standardize-high-bandwidth-flash/
SanDisk Develops HBM Killer: High-Bandwidth Flash (HBF) Allows 4 TB of VRAM for AI GPUs | TechPowerUp, accessed on August 8, 2025, https://www.techpowerup.com/332516/sandisk-develops-hbm-killer-high-bandwidth-flash-hbf-allows-4-tb-of-vram-for-ai-gpus
Sandisk to Collaborate with SK hynix to Drive Standardization of High-Bandwidth Flash Memory Technology, accessed on August 8, 2025, https://www.sandisk.com/company/newsroom/press-releases/2025/2025-08-06-sandisk-to-collaborate-with-sk-hynix-to-drive-standardization-of-high-bandwidth-flash-memory-technology
Overview Guide — GPUDirect Storage Overview Guide, accessed on August 8, 2025, https://docs.nvidia.com/gpudirect-storage/overview-guide/index.html
GPUDirect Storage: A Direct Path Between Storage and GPU Memory - NVIDIA Developer, accessed on August 8, 2025, https://developer.nvidia.com/blog/gpudirect-storage/
What is GPUDirect Storage? | WEKA, accessed on August 8, 2025, https://www.weka.io/learn/glossary/gpu/what-is-gpudirect-storage/
The Micron - 9400 NVMe SSD Performance With NVIDIA - Magnum IO GPUDirect - Storage Platform, accessed on August 8, 2025, https://www.micron.com/content/dam/micron/global/public/products/white-paper/micron-9400-nvidia-gds-vs-comp-white-paper.pdf
Accelerate AI and ML workloads with OCI, NVIDIA Magnum IO GPUDirect Storage, and IBM Storage Scale - Oracle Blogs, accessed on August 8, 2025, https://blogs.oracle.com/cloud-infrastructure/post/accelerate-ai-ml-workloads-oci-nvidia-ibm
GPUDirect Demystified: Why Your File System is Crucial for Maximum GPU Throughput & Efficient AI Data Storage - Hammerspace, accessed on August 8, 2025, https://hammerspace.com/gpudirect-demystified-why-your-file-system-is-crucial-for-maximum-gpu-throughput-efficient-ai-data-storage/
Samsung Electronics Introduces Industry's First 512GB CXL Memory Module, accessed on August 8, 2025, https://news.samsung.com/us/samsung-electronics-introduces-industrys-first-512gb-cxl-memory-module/
Press Room - Compute Express Link, accessed on August 8, 2025, https://computeexpresslink.org/news/
CXL™ Consortium Board of Directors – Statements of Support - Compute Express Link, accessed on August 8, 2025, https://computeexpresslink.org/wp-content/uploads/2024/01/CXL_2.0-Launch-Statements-of-Support_FINAL.pdf
Products - ASTERA LABS, INC., accessed on August 8, 2025, https://www.asteralabs.com/products/
Leo CXL® Smart Memory Controllers - ASTERA LABS, INC., accessed on August 8, 2025, https://www.asteralabs.com/products/leo-cxl-smart-memory-controllers/
Lenovo ThinkSystem SR650 V3 Server Product Guide, accessed on August 8, 2025, https://lenovopress.lenovo.com/lp1601-thinksystem-sr650-v3-server
The Current State Of CXL Support On Linux - Phoronix, accessed on August 8, 2025, https://www.phoronix.com/news/Linux-6.11-CXL
Run Modern, AI, and Traditional Apps Better with vSphere in VMware Cloud Foundation 9.0, accessed on August 8, 2025, https://blogs.vmware.com/cloud-foundation/2025/06/17/run-modern-ai-and-traditional-apps-better-with-vsphere-in-vcf-9-0/
Boost VMware vSphere 8 U3 Performance with Memory Tiering - StarWind, accessed on August 8, 2025, https://www.starwindsoftware.com/blog/improve-server-consolidation-with-vmware-vsphere-8-u3-memory-tiering-feature/
GPUDirect Storage support for IBM Storage Scale, accessed on August 8, 2025, https://www.ibm.com/docs/en/storage-scale/5.2.2?topic=architecture-gpudirect-storage-support-storage-scale
Amazon FSx for Lustre now supports Elastic Fabric Adapter and ..., accessed on August 8, 2025, https://aws.amazon.com/about-aws/whats-new/2024/11/amazon-fsx-lustre-elastic-fabric-adapter-nvidia-gpudirect-storage/
NVIDIA Grace Hopper Superchip Architecture Whitepaper, accessed on August 8, 2025, https://resources.nvidia.com/en-us-grace-cpu/nvidia-grace-hopper
CMM-D | CXL Memory | Samsung Semiconductor Global, accessed on August 8, 2025, https://semiconductor.samsung.com/cxl-memory/cmm-d/
CXL 3.0 and the Future of AI Data Centers | Keysight Blogs, accessed on August 8, 2025, https://www.keysight.com/blogs/en/tech/digital-test-instruments/2024/07/10/cxl-3-0-and-the-future-of-ai-data-centers
Redefining Possibilities in Memory Technology: Samsung CXL Memory Appliance with Orchestration Console | Samsung Semiconductor Global, accessed on August 8, 2025, https://semiconductor.samsung.com/news-events/tech-blog/redefining-possibilities-in-memory-technology-samsung-cxl-memory-appliance-with-orchestration-console/
Compute Express Link (CXL): All you need to know - Rambus, accessed on August 8, 2025, https://www.rambus.com/blogs/compute-express-link/
System Composability Using CXL, accessed on August 8, 2025, https://www.openfabrics.org/wp-content/uploads/2024-workshop/2024-workshop-presentations/session-10.pdf
CXL Memory Pool Appliance Market Research Report 2033 - Dataintelo, accessed on August 8, 2025, https://dataintelo.com/report/cxl-memory-pool-appliance-market