Cloud native EDA tools & pre-optimized hardware platforms
Enabling debugging for an FPGA-based prototype requires some effort, but yields high return on investment, which is often only discovered when it is too late. Prototypers focus on getting the bit files for their FPGAs ready as quick as possible, forgetting about plan B (i.e. how to troubleshoot the prototype), which will be needed when things go wrong. In this article we will describe how the HAPS integrated prototyping solution enables use case specific debug capabilities for bring-up and immediate use of a prototype from the project start.
A debug solution for FPGA-based prototypes needs be capable of addressing the specific needs for a whole variety of defects, in context of multiple use cases. Defects may be introduced during the prototype bring-up, such as when making the design FPGA ready, or even when setting up the FPGA hardware. Defects may be hiding in the original ASIC design itself, where prototype based verification is a means to identify them, via directed software tests or real world scenarios. Furthermore, defects may be contained in the embedded software such as firmware, OS kernel, drivers or higher level software.
Different user personas have varying technical backgrounds and as a result they also have different requirements on the accessibility and usability of a debug solution. The use of prototyping is typically very early in the design/development stage for both, hardware and the software. It is especially important that the debug solution is able to establish a common ground for prototypers, hardware designers and software developers to co-debug defects which cross the hardware and software boundary. A full waveform trace of every SoC’s RTL signal and register is of little help for the software developer, just like a Linux kernel core dump is of little help for the hardware designer. But, there is information in the software traces which can certainly provide the hardware designer with a better understanding of a particular scenario in order to narrow down the problem. And, there are certain signals and registers that are of high interest for the software developer to understand the hardware’s response to the software, which will enable more efficient and focused debugging.
This article introduces HAPS Deep Trace Debug (DTD), which is one of several HAPS debug solutions. HAPS DTD supports multiple use cases and allows tailoring the visibility to the prototype user persona, either hardware or software. HAPS DTD falls into the category of the so-called embedded trace extraction techniques. Here, signals in the user’s design are probed using dedicated instrumentation logic and stored within a sample buffer for offline post-process analysis.
Figure 1: HAPS debug technology ecosystem
This is in contrast to real-time debug (RTD) where signals are probed and viewed just in time using lab equipment such as a logic analyzer or oscilloscope. Both of those trace-based debugging techniques have their application areas and are supported by HAPS. Real-time debug is required when analyzing hardware interfaces and protocols (e.g. PHYs) and is enabled by HAPS through specific Logic Analyzer Interface (LAI) daughter boards. RTD even allows correlation of the interfaces with the user’s logic.
Deep Trace Debug is enabled for the user’s design on the HAPS system using HAPS ProtoCompiler. HAPS ProtoCompiler prototyping software is tailor made for HAPS systems, providing prototypers with an integrated prototyping solution. HAPS DTD scales with HAPS system capacity, which ranges from a single FPGA to 64 FPGAs using 16 HAPS-80 systems, approximately 1.6 billion ASIC gates. HAPS ProtoCompiler’s hardware awareness allows for the automation of many tedious and error prone bring-up tasks such as gated clock conversion, partitioning and the insertion of the required interconnect and time domain multiplex communication logic. Moreover, the tool automates the instrumentation of the user’s design for debug and connecting the debug logic to the HAPS system’s hardware infrastructure such as connectors, memories and debug interfaces.
This is especially important when the design is partitioned over multiple FPGAs and has multiple clock domains as it imposes additional demands on the trigger and sampling logic for tracing. With manual flows and non-integrated prototyping these are major obstacles as they are hard to deal with. Often, the required effort prevents prototypers from investing in debugging as they are primarily focused on getting the FPGA bit files in place. However, frustration and many lengthy prototype bring-up iterations typically remind prototypers of how expensive not having debug can be, as it is not easy to bring back debug into the prototype when using a manual flow. Every change to the design, as well as additional debug instrumentation logic may impact the partitioning with a whole ripple effect on redesigning the communication and interconnect. Through the automation in HAPS ProtoCompiler, the preparation of a HAPS system for debug almost comes at no cost in terms of enablement effort except for the manual steps to tailor the debug visibility to meet the user’s needs for a given design and use case. This is accomplished using the HAPS ProtoCompiler Instrumentor and will be explained later in this article.
For DTD, performance, signal capacity and tracing depth are parameters which the prototype can adjust to find the optimal trade-off for the targeted debugging task. For interface IP prototypers, performance may be of highest importance due to the real-time requirements of the analog interfaces. For SoC prototypers, it may be much more important to capture a large amount of signals (e.g. the interconnect) over a long scenario, but with less strict requirements on performance.
This flexibility is typically not possible with “traditional” FPGA Block RAM (BRAM) based tracing. Here, the tracing depth is strictly limited by the available BRAM and BRAM for debug now even competes with the BRAM required for the design. Therefore, next to BRAM based tracing, HAPS DTD allows tracing into external DDR3 memory banks. This doesn’t take away any memory from the design and with 16GB of DDR3 storage for sampling data, the prototype can be very generous in terms of defining the signals scope for sampling. With DTD, the prototyper can choose to sample up to 2000 signals per FPGA at 20Mhz, or 250 signals per FPGA can be traced at 160MHz in case performance is more important than signal breadth (e.g. interface IP).
As a frontend to the whole DTD flow, the prototyper uses the HAPS ProtoCompiler Instrumentor. Here, the prototype tailors the visibility breadth and depth to the needs of the user persona. As a first step, an IICE (intelligent in-circuit emulator) is instantiated. The IICE will later serve as the communication and control interface between the HAPS ProtoCompiler Debugger on the host and the instrumented design on HAPS. An IICE is associated to a clock domain in the design. If multiple clock domains need to be traced, multiple IICEs can be inserted. Furthermore, per IICE the prototyper choses the sampling memory (BRAM, DDR on board, DDR on debug hub). Now, the instrumentation can be conducted. A key value of the HAPS ProtoCompiler instrumentation flow is the ability to define signals for sampling in the original RTL code.
Figure 2: Instrumentation in the original
VDHL/Verilog code
Figure 3: Post compile instrumentation
Instead of trying to find signals back in generated files such as EDIF (Electronic Data Interchange Format) with generated signal names, the instrumentation is conducted in the original VHDL or Verilog implementation. In this implementation, the user can define breakpoints to be used as sampling triggers. Breakpoints allow for very complex trigger conditions at a small cost in area. Another type of trigger is a watchpoint. Here the instrumentation logic is associated with an RTL node or signal. Within the IICE the prototype also defines the complexity of the trigger logic. HAPS ProtoCompiler allows for complex state machine based triggers with multiple conditions. This is very beneficial to pin-point tracing to scenarios of interest by only starting the trace after e.g. specific registers have been written in a specific sequence with specific values. An example for this is when a peripheral shall be traced after being configured by the software (e.g. 1. reset has been applied, 2. control register enable bit has been written, 3. interrupts have been enabled). All triggers can be pre-armed so that a software developer would not even need to interfere with the HAPS ProtoCompiler RTL Debugger. Furthermore, it shall be mentioned that HAPS ProtoCompiler allows for post compile instrumentation which is crucial as RTL may not be always usable for instrumentation. As an example, the VHDL/Verilog code may contain a for-loop and only in the fully elaborated circuit the user can then select the signals within a schematic view. All the steps for instrumentation are fully scriptable, as also all other phases in the prototyping flow are with HAPS ProtoCompiler.
Figure 4: Scripted instrumentation for HAPS ProtoCompiler
The triggers and samples defined during the instrumentation stage can by dynamically activated in the HAPS ProtoCompiler RTL Debugger. The HAPS ProtoCompiler RTL Debugger is the counterpart of the HAPS ProtoCompiler Instrumentor. It is used with the instrumented design on the actual HAPS system. Breakpoints and watchpoints which have been defined can be dynamically activated to cause the trigger events within the IICE on the HAPS system. These triggers then cause data to be sampled.
Two forms of watchpoint triggers are supported: value and transition. A value watchpoint triggers when the watched signal attains a specific value. A transition watchpoint triggers when the watched signal has a specific value transition. Multiplexed groups of instrumented signals defined in the instrumentor can be individually selected for activation in the debugger. Cross triggering allows the trigger from one IICE unit to be used to qualify a trigger on another IICE unit, even when the two IICE units are in different time domains. As an advanced trigger mechanism a runtime programmable state machine allows you to specify the temporal and logical behavior that combines trigger conditions into a complex trigger function. The waveform display control displays the sampled data in a waveform style. By default, this feature uses the Synopsys DVE waveform viewer but also GTK Wave or Synopsys Verdi nWave are supported.
In summary, HAPS integrated prototyping and its debug solution addresses the needs for efficient prototype bring-up and use. The HAPS debug ecosystem embraces software debugging and trace based debugging of the hardware. Using Deep Trace Debug and the HAPS ProtoCompiler Instrumentor flow, the debugging capabilities can be tailored to the requirements of the user and the characteristics of the design under test. Through the automation of HAPS ProtoCompiler, there is almost no overhead to enable debug, even for large and complex multi-FPGA based prototype with multiple clock domains.