For Memory Expansion and Memory Pooling

CXL® (Compute Express Link®) is an industry standard, open protocol for high speed and low latency communications between host accelerator, which are increasingly used in emerging applications, such as Artificial Intelligence (AI) and Machine Learning (ML).

SMART Modular, along with other industry leaders, such as Alibaba, Cisco, Dell EMC, Facebook, Google, Hewlett Packard Enterprise, Intel Corporation and Microsoft have teamed up to form an open industry standard group to develop technical specifications that facilitate breakthrough performance for emerging usage models while supporting an open ecosystem for data center accelerators and other high-speed enhancements. 

CXL Protocols

The CXL standard defines 3 protocols that are dynamically multiplexed together before being transported via a standard PCIe 5.0 PHY at 32 GT/s:

• Native Widths: x16, x8, x4; x16 partitionable: x16, 2x8, 4x4, x8 + 2x4; x8 partitionable: x8, 2x4; degraded widths: x2, x1.

• Data Rate: 32 GT/s.

• Degraded support: 8 GT/s and 16 GT/s (128b/130b).

• Independent coherency models like CXL.cache, CXL.mem to support variety of use cases in Computational Storage, Network Acceleration and Persistent Memory.

• Plug and Play: either a PCIe or a CXL card/device can be plugged in.

• Reuse PCIe® PHY, channel, circuit, Retimer, etc.

CXL is Designed to Support Three Primary Device Types:

Example: Type-1 CXL Device

SMART NIC or video accelerators, which run specific algorithm function like video transcoding on data present in Host Memory.

Example: Type-2 CXL Device

FPGA or CPU based accelerators with integrated memory (like HBM or DDR).

Example: Type-3 CXL Device

Persistent Memory or Computational Storage devices sitting on CXL bus. Such devices cannot access Host Memory on their own.

CXL Memory Expansion

SMART Modular develops CXL Type 3 (CXL.mem) memory products to address the industry’s need for more memory per processor core. This approach allows for a more flexible and scalable memory architecture, where memory devices can be added or removed as needed, without the need to replace or upgrade the entire system.

Lower Total Cost of Ownership (TCO)

SMART's CXL AIC is a more economical choice for expanding memory under current system architectures. Instead of purchasing 8 128GB DDR5 RDIMMs, clients can achieve equivalent memory capacity by employing 8 64GB DDR5 RDIMMs in conjunction with an 8-DIMM CXL Add-In Cards (AICs). This configuration results in a remarkable reduction in costs, amounting to 60%.

CXL Memory Pooling

CXL 2.0 supports switching to enable memory pooling for efficient memory allocation. At 2.0 level, device can be partitioned as Multiple Logical Devices (MLD), allowing up to 16 hosts to simultaneously access different portions of the memory.

As an example, Host 1 (H1) can use half the memory in Device 1 (D1) and a quarter of the memory in D2 to finely match the memory requirements of its workload to the available capacity in the memory pool. The remaining capacity in D1 and D2 can be used by H2-H#.

Key Features for CXL AIC:

• Available in type 3 PCIe Gen5 Full Height, Half Length (FHHL) PCIe form factor.

• The 4-DIMM AIC (CXA-4F1W) accommodates four DDR5 RDIMMs with a maximum of 512GB of memory capacity when using 128GB RDIMMs, and the 8-DIMM AIC (CXA-8F2W) accommodates eight DDR5 RDIMMs with a maximum of 4TB of memory capacity.

• The 4-DIMM AIC uses a single CXL controller implementing one x16 CXL port while the 8-DIMM AIC uses two CXL controllers to implement two x8 ports, both resulting in a total bandwidth of 64GB/s.

• The CXL controllers support “Reliability, Availability, and Serviceability” (RAS) features, and advanced analytics.

• Both offer enhanced security features with in-band or side band (SMBus) monitoring capability.

• To accelerate memory processing, these add-in cards are compatible with SMART’s Zefr™ ZDIMMs™.

Key Features for CMM-E3S:

• Available in EDSFF E3.S 2T (2U Short) form factor

• CXL-2.0 compatible with PCIe Gen5 speeds running at 32GT/s

• Available in 64GB and 96GB densities

• Supports Reliability, Availability and Serviceability (RAS) features added in CXL 2.0

• Powered by only 12V supply from EDSFF compatible edge interface (SFF-TA-1009)

• Supports sideband interfaces for real-time debug, management and system update, enabling out-of-band management of the module

• Supports additional security features to protect data from side channel attacks