An Introduction to the CXL Device Types

Compute Express Link (CXL) is the latest specification in interconnect technology for high bandwidth devices. It provides high-speed, efficient connectivity from CPUs to other components of the high-performance computing platform.

CXL’s coherent memory access capability between a host CPU and a device, such as hardware accelerators, meets the requirements for handling data and compute intensive workloads in next-generation designs by leveraging the advanced capabilities of PCIe architecture.

Let’s explore the types of CXL devices and the verification challenges that are unique to CXL, like maintaining the cache coherency between a host CPU and an accelerator. For an initial overview of this specification, check out our previous blog here.

Type 1 CXL Device

• Implements a fully coherent cache but no host-managed device memory
• Extends PCIe protocol capability (for example, Atomic operation)
• May need to implement a custom ordering model
• Applicable transaction types: D2H coherent and H2D snoop transactions

Type 2 CXL Device

• Implements an optional coherent cache and host-managed device memory
• Typical applications are devices which have high-bandwidth memories attached
• Applicable transaction types: All CXL.cache/mem transactions

Type 3 CXL Device

• Only has host-managed device memory
• Typical application is a memory expander for the host
• Applicable transaction types: CXL.mem MemRd and MemWr transactions

A CXL.cache/mem design has to maintain cache coherency between the host and device agent caches and their corresponding memory. The Bias Based Coherency model for a Type 1 CXL device defines two states for device attached memory – the host bias and the device bias. Each possess their own verification challenges.

CXL acts as a high-performing I/O interconnect system, trying to ensure reliable and efficient access to memory distributed throughout various components. Some of these components optimize performance by making use of the local cache and reduce the overhead of memory access. To support this type of configuration, CXL.cache protocol ensures that data held across the components in either memory or local cache remain coherent and consistent to each component. Device components in CXL are typically used as accelerators for computationally intensive applications, and hence contain a local cache. So, if the host component wants to access the same location of memory, it can then force the device to evict the line from its local cache and update the corresponding memory location depending on the cache line state.

The CXL.cache protocol defines interaction between the device and host as a number of requests that each have at least one associated response message and sometimes a data transfer. The interface consists of three channels in each direction: Request, Response, and Data. The channels are named for their direction – D2H (Device to Host) and H2D (Host to Device).