Advanced functional verification techniques like constrained-random test generation are well established for ASICs, but not so much for FPGAs. At a recorded Cadence Theater presentation at the 2013 Design Automation Conference, Torsten Kӧnig, lead verification architect at Siemens Healthcare, showed how he convinced his company's management to try the Cadence Specman environment for an FPGA project, and he presented the results his team achieved.
The presentation was titled "Musings on Advanced Functional Verification with Specman/e in FPGA for Medical Devices." Kӧnig began by presenting some of the excuses he heard when he first tried to bring advanced functional verification into the FPGA environment. These included the following statements (with some responses):
"Our design is much smaller than an ASIC, so why bother?" (Can you handle the complexity?)
"We can't afford to re-verify all legacy designs." (You don't have to.)
"Directed test cases are faster and fully sufficient for our designs." (Again, can you grasp the full complexity of the design?)
"It's quicker to simply try it out in the lab." (It could take weeks to find a bug.)
"If we miss a bug, we'll simply build a new bitstream." (And since this is a medical device, you'll have to get FDA approval for that.)
To convince management that FPGA verification is truly complex, Kӧnig put together a few numbers:
- A 256-bit memory has 2256 unique combinations
- Roughly, this amounts to 1076 unique combinations
- If we want to check at 1 GHz that every combination works as expected, it will take 1067 seconds or 3.1*1059 years
- The Sun has existed for 4.5*109 years
- A Xilinx Virtex 7 2000T contains 46 Mbytes of block RAM. This yields about 10111,600,000 unique combinations
In short, Kӧnig said, you have better odds of winning the German lottery (1 in 140 million) than completely verifying a complex FPGA device with traditional methods.
A Trip to Paris
In addition to these rather startling numbers, Kӧnig used a hypothetical trip to Paris to help management understand what constrained-random stimulus is, and why this powerful Specman feature should be used in an FPGA environment.
There are two analogies. In the first, echoing a traditional verification approach, you go to your hotel and plan routes to the sights you want to see. This is equivalent to writing directed test cases. If streets are blocked, the routes are no longer valid - this is like FPGA modifications that render test cases invalid. You see the sights you had planned to see - but you miss a number of interesting things you didn't know about. Likewise, traditional directed-test methodologies miss hidden bugs because no one is looking for them.
The second analogy uses a coverage-driven, constrained-random approach. First, you look at a map and list the sights you want to see (coverage). Next you start walking and randomly choose a direction at each corner, making sure you stay in inner Paris (constrained-random). You collect a ticket from every sight you visit, and you're not done until you have tickets for all the selected sights (automatic regression). It doesn't matter if some streets are blocked, and you see far more sights than you had planned to see (hidden bugs are being found).
Trying a New Approach
Management gave Kӧnig the go-ahead to try constrained-random stimulus with Specman on an image-chain design project. In this device, a detector provides data (medical imagery), a memory manager processes data and places it in memory, and software acquires the data from memory.
Early in the simulation, engineers found a deadlock condition. They were able to determine that a bit flip in another module caused the RAM controller to hang. "Due to the functional verification from Specman, we found this at the source of the problem," Kӧnig said. "We didn't have to dig deep into the controller to find out why this was happening." The team also found some processing unit FIFO errors.
Kӧnig was also able to show that Specman makes it easy to reuse verification components. He used the example of a RAM controller from an old project. The controller was known to have a complicated bug that originally took weeks of lab work to find. The old controller was hooked up to the crossbar interface from the new project, and it only took two or three of the Specman test cases - with no adaptation -- to find the bug.
Kӧnig cited the following project results:
- Around 80 functional bugs were found with Specman/e
- The investment for this project was recovered more than four times over
- Changed use cases during development did not impact the team
- Not a single bug slipped through to integration test
To listen to the Siemens audio presentation and view the slides, click here. The presentation was given at 5:00 p.m. Tuesday June 4.