CCIX Over PCIe: Faster Coherent Interconnects

VIP Expert

Oct 10, 2018 / 2 min read

Next generation SoC designs require faster coherent interconnects for high performance applications such as machine learning, network processing, storage off-load, in-memory data base and 4G/5G wireless technology. CCIX (Cache Coherent Interconnect for Accelerators), a new protocol standard, provides benefits of cache coherency and peer processing which enables the faster interconnect. CCIX is designed smartly to use the well-established PCIe infrastructure to carry coherency packets across the link with little modification. CCIX specification is compatible with PCIe base specification 4.0. PCIe implementation is extended to implement a CCIX transaction layer, responsible for carrying the coherency messages.

ccix block diagram

Data transfer above 16GT/s:

Per the CCIX Consortium, data transfer can be done at a maximum speed of 25GT/s, increasing the PCIe 4.0 maximum speed, which is otherwise limited to 16 GT/s. CCIX introduced a feature called Extended Speed Mode (ESM) to achieve 20 GT/s and 25 GT/s link speed. PCIe components supporting this mode must enhance the physical layer to achieve higher speed; however, no enhancement is required in ordered sets fields except EIEOS and Control SKP OS format change. To achieve the CCIX ESM speed, below steps are performed maintaining compatibility between PCIe and CCIX protocols.

  1. Perform link up as per highest mutually supported PCIe speed with normal PCIe link initialization process
  2. Application layer reads CCIX-specific configuration registers to check if both components support ESM
  3. If ESM is supported, then CCIX-specific registers on both components are programmed to map PCIe link speed as mentioned in the below table
  4. Link is retrained to achieve CCIX ESM speed, so that both components can perform data transfer at highest rate of 25GT/s
ccix table faster coherent interconnects

The Verdi snapshot below displays link up at 25GT/s with Synopsys VIP for PCIe 4.0. The required bit period is 40ps on the serial PCIe link.

verdi snapshot ccix over pcie

Higher performance can be achieved using PIPE 4.4.1 (PHY interface for PCIe), which reduces the latency in SerDes, and thereby leads to a faster simulation. It also enables easy and fast integration. However, some customization is required, as currently there is no industry standard specification for CCIX over PIPE interface. Message Bus Interface helps to do the required customization. To learn more about it, read our recent blog –  PCIe PIPE 4.4.1: Enabler for PCIe Gen4.

Synopsys is working with early adopters of CCIX and PCIe 5.0/4.0. A combination of Synopsys VIP and DesignWare IP controller for PCIe 4.0 provides a complete solution for CCIX interconnect design and verification. The VIP for PCIe 4.0 provides flexibility to work with CCIX specified configurable features along with robust protocol checks, simplifying the verification of CCIX and PCIe protocols with different types of complex configurations.

Continue Reading