Cadence.com will be under maintenance from Friday, Oct. 3rd at 6pm (PST) thru Sunday, Oct 5th at 11pm (PST).
Cadence.com login, registration, community posting and commenting functionalities will be disabled.
Home > Community > Blogs > Digital Implementation > the case for the tiny testcase
 
Login with a Cadence account.
Not a member yet?
Create a permanent login account to make interactions with Cadence more convenient.

Register | Membership benefits
Get email delivery of the Digital Implementation blog (individual posts).
 

Email

* Required Fields

Recipients email * (separate multiple addresses with commas)

Your name *

Your email *

Message *

Contact Us

* Required Fields
First Name *

Last Name *

Email *

Company / Institution *

Comments: *

The Case for the Tiny Testcase

Comments(1)Filed under: Digital Implementation, verification, encounter, testcase, test, test case, debug, tiny testcase, small testcase, Bob Dwyer
I often joke with customers that, although I realize they have to work on large designs, I do my best work on designs with just 2 or 3 instances. That's because I'm often trying to replicate an issue they've observed on their design and I'm attempting to reproduce that behavior in a smaller circuit. I've found tiny testcases to be extremely efficient ways to gain quick clarity on tool behaviors which can then be more effectively applied to the real design. But it's not just having a tiny testcase that's most useful. It's the act of creating the small testcase where most insight is gained.

But it's not always easy to create a tiny testcase. Do you know how to write a syntactically correct gate-level Verilog netlist from scratch with a text editor? Do you know how to contrive complicated timing scenarios? Do you know to modify .lib/LEFs to replicate the things likely present in a design in a smaller setting? These are non-trivial things for someone who works in many different aspects of a design flow as designers are tasked with understanding an ever-increasing breadth of tools.

I recall a quote from a colleague of mine, Thad McCracken (interview here), a few years ago. We were working on a benchmark and were having a hard time reproducing some of the issues we were observing in a small testcase. He said "If we can't reproduce it in a small testcase it tells me we fundamentally don't understand the problem well enough."

Isn't that the truth of it? With the exception of run-time/memory related issues, nearly every issue we run into can be replicated in a tiny representative testcase -- but doing so can be very difficult.

Customers often joke with me that the oldest tricks in the EDA Applications Engineering books are (1) to ask if you're using the latest tool version and (2) ask for a testcase. Sure it's easy to get a self-contained testcase with Encounter (see the link for saveTestcase at the end of this post). But a better question to ask is, "Do you have any hunches on the likely circumstances causing the issue?"

Tiny testcases can be powerful in the user community as well. Some of the most disciplined and effective project teams and CAD groups I've worked with relentlessly stress the tool at the onset of a project in ways they're going to need it to perform during crunch time. Sure, some things only come to light in the context of real designs. But creating tiny testcases can efficiently flush out the gaps between project requirements and tool capabilities to give everyone a chance to find ways to get the software to do what it needs to do to meet project requirements.
 
In our busy schedules we often feel like we don't have time to create small testcases to triage a situation. And indeed, sometimes we just need to send a testcase in to R&D for resolution. But next time you run into an issue I'd encourage you to consider whether a tiny testcase might shed more light on the situation.

To borrow a line from Paul Cunningham: "Sometimes we need to slow down to speed up."
 
-Bob Dwyer

Related Reading:

Comments(1)

By bandit on November 27, 2012
Spot on. Doesn't matter the kind of programming you are doing or even the kind of machine.
This is a case for good design. Take the time when the cost of a mistake is the cheapest.
For example, C. The standard in several cases state basically it depends on what the compiler writer wants to do. In most cases, you should do things where it doesn't matter, but every so often it does. Write a simple test case. And DOCUMENT in the code why you had to care (please!!!)

Leave a Comment


Name
E-mail (will not be published)
Comment
 I have read and agree to the Terms of use and Community Guidelines.
Community Guidelines
The Cadence Design Communities support Cadence users and technologists interacting to exchange ideas, news, technical information, and best practices to solve problems and get the most from Cadence technology. The community is open to everyone, and to provide the most value, we require participants to follow our Community Guidelines that facilitate a quality exchange of ideas and information. By accessing, contributing, using or downloading any materials from the site, you agree to be bound by the full Community Guidelines.