Functional verification entails changing the state of a logic design and measuring that the response generated by the design is correct. Verification environments change the state of designs by driving stimulus in the form of directed or constrained random inputs.
Regardless of stimulus type, metric-driven verification (MDV) is about data-driven decision making providing clear metrics that are easily understood, to improve verification predictability, productivity, and quality. MDV is a closed-loop process, starting with executable verification plans (vPlan), to testbench creation, to execution, and finishing with measurements and analysis to see what else is left to be done (Figure 1).
The MDV solution utilizes the most powerful specific purpose verification plan authoring software, vPlanner™ tool, which is an integral part of the Cadence® Incisive® vManager™ solution. The MDV flow starts with automated planning, either annotating existing specifications with verification intent or creating an executable verification plans using team member inputs captured in a spreadsheet. The plan specifies the verification environment requirements to be implemented in the testbench to ensure design intent can be delivered. Verification IP (VIP) provides immediate access to the plan-based MDV methodology by delivering a protocol-specific verification plans and test suite. Verification plans, or vPlans, become executable once embedded within the Incisive vManager environment, and are tightly coupled to the test execution, as well as maintaining which tests provided which result. Plans can be embedded within plans, and can be parameterized for plan re-use, which improves verification productivity.
MDV spans from simple directed test methodologies, thru to register-transfer level (RTL)-based coverage driven and software driven approaches. For RTL verification, the Universal Verification Methodology (UVM) standard has been specifically designed to provide powerful constrained random testing. The power of constrained random stimulus is that vectors are automatically generated without having verification engineers spell out each set of input vectors manually. With the constrained random stimulus come constrained random changes in state. In this environment, it is imperative that the verification team is able to specify which states have been explored, and that for those explored states, the correct response to stimulus vectors is observed. For this reason, functional coverage is utilized. Functional coverage is defined for and captures scenarios that are important for the verification environment to observe and capture. As such, functional coverage serves as a guide in the process of determining the state of the verification effort and when that effort has been completed to satisfaction of the relevant stakeholders. Coverage is a tangible measurement of all the features and functions being verified, and is a predictive metric.
Verification simulation, formal analysis, analog mixed-signal, acceleration, and emulation execution engines are dispatched and the convergence of results from multiple verification engines is managed within the Incisive vManager environment. Failures are analyzed and prioritized. Regressions and coverage is analyzed and changes to the testbench are developed to reach closure. Execution optimization techniques are necessary so that results are available for engineers to work on without waiting for execution to conclude.
MDV is used to measure real-time progress with on-demand information, as well as to determine when high-quality verification closure is achieved. Coverage, checks, and assertions provide the verification-specific metrics used to determine closure. Verification job failures, bugs, and design revisions all provide insight into the actual status of a project. Progress analysis and reports help the verification team make adjustments to their resource allocation (people and tools) to reach closure more efficiently and measure closure more accurately.
MDV allows an engineer to capture all of the key metrics in the verification process, from the coverage notions described previously for the device under test (DUT metrics), to the project metrics necessary to be visible and managed (Figure 2). The list of metrics available is endless, yet the MDV flow provides an executable verification plan to organize, manage, and view this overwhelming amount of metrics in a form that is human readable and understandable.
- Improved verification planning results in higher quality
- Metrics analysis and reporting increase schedule predictability
- Failure/metrics analysis and job automation boost overall team productivity