The Journey of Coverage Closure

Taruna Reddy

Sep 10, 2024 / 2 min read

As reported in the annual Synopsys Global User Survey, coverage closure has consistently been highlighted in the top challenges verification teams struggle to overcome.  In 2023, it’s ranking rose into the Top-5 challenges for verification teams. One of the reasons being that coverage closure requires a different approach depending on the phase in the project cycle. Read on to see how our customers have used Synopsys VCS ICO (intelligent coverage optimization) and VSO.ai solution to overcome these challenges. 

Testbench Stabilization

In the early phase of a project, the primary goal is testbench stabilization, faster root-cause analysis, and uncovering bugs as the RTL and testbench are evolving. At this stage, there is probably still no coverage being collected. Apart from functional correctness, the simulator should be able to provide these insights by leveraging AI/ML to improve the stimuli diversity. The increased stimuli distribution often exposes issues like over/under-constraints in the testbench which could be preventing some coverage being hit.  This is one of the primary goals of ICO in VCS. 

Customer Use Case: Microsoft implemented VCS ICO, leveraging its AI/ML technology to enhance verification efficiency and uncover bugs in just 300 seeds versus the traditional 10,000+, as well as demonstrated six new error signatures, ultimately facilitating a 'left shift' in the verification cycle. 

Qualcomm also used VCS ICO to achieve these goals in the early phase, all the way up to the last stages of the project.

Bug Hunting

In the intermediate phase of a project, the RTL is more stable and therefore functional coverage is expected to start an upward trend along with the bug rate. The goal in this stage is to find corner-case bugs and improve the regression turn-around time (TAT) to reach coverage faster. Improving the regression TAT is critical to enable running more regressions with the minimum compute resources needed to uncover more bugs. Therefore, utilizing solutions like VSO.ai that leverage AI/ML technology to learn from history on the highest coverage contributing tests and running more of them will help improve the regression TAT. 

VCS’ sequential constant analysis functionality automatically searches for unreachable coverage targets for line, condition, toggle and branch coverage, and removes them from the list of coverable objects can be turned on in this stage. 

Customer Use Case: Using the Synopsys AI-driven verification solution VSO.ai, NVIDIA was able to demonstrate a regression TAT reduction of 2-5X and up to 16X at AMD

Coverage Signoff

In the last phase of a project, also known as the ‘stable phase’, the emphasis is on closing the remaining coverage and coverage holes which is typically a very manual process of creating directed tests. The unreachability analysis (UNR) in VCS, which is tightly integrated with Synopsys VC Formal verification solution, can help reduce verification efforts ranging from 8% to 80%.

Customer Use Case: Cisco used UNR in VCS and observed a 9% improvement in coverage, eliminating noise and allowing the team to focus on the real coverage holes.

 

VSO.ai can also help boost coverage depending on the degree of randomness and connections established in the testbench. The root-cause analysis insights on the illegal and non-sampled bins provided in VCS ICO and VSO.ai help to close the coverage holes closer to tape-out, reducing the number of directed tests that need to be written. 

Customer Use Case: NVIDIA implemented VCS ICO, VSO.ai and VCS UNR depending on the phase of the project to reduce compete resources, left-shift coverage by at least one milestone, and find unique bugs earlier.

 

Conclusion

All in all, coverage closure is a high-value problem and typically requires the use of multiple technologies within the simulator leveraging AI/ML and sometimes across solutions like UNR. 

Continue Reading