Home > Community > Blogs > Functional Verification > beyond coverage adding arbitrary metrics to your metric driven flow
 
Login with a Cadence account.
Not a member yet?
Create a permanent login account to make interactions with Cadence more conveniennt.

Register | Membership benefits
Get email delivery of the Functional Verification blog (individual posts).
 

Email

* Required Fields

Recipients email * (separate multiple addresses with commas)

Your name *

Your email *

Message *

Contact Us

* Required Fields
First Name *

Last Name *

Email *

Company / Institution *

Comments: *

Beyond Coverage: Adding Arbitrary Metrics To Your Metric-Driven flow

Comments(0)Filed under: Functional Verification, Enterprise Manager, metric driven verification (MDV), Incisive Enterprise Simulator (IES), e, Specman, IES-XL, vPlan

The most common metrics used in current metric driven verification (MDV) flows are various forms of code coverage (block, FSM, toggle etc.), assertion coverage, and functional coverage.  As many of you know, all of these can be collected into Enterprise Manager for individual analysis, merged together into a single unified coverage score, and then ranked to easily identify the most effective elements of the testbench.

Over the years there have been various requests to support other metrics in this flow, i.e. to take advantage of the automation infrastructure that’s already been built for verification purposes and apply it to track project management-oriented, “non-electrical”, operational data.  Examples include verification code statistics (amount of assertions developed over time, amount of cover groups/items implemented, amount of code developed), server farm utilization per day, and many others.

Fortunately, the capabilities to support this more expansive version of a metric-driven flow are already built-in to the Incisive platform!  Specifically, you can import all this non-electrical metric data into a high-level verification language (HVL) via its interfaces/APIs (or even file I/O), and transform the data into functional coverage points.  Rephrasing: you can model the required data characteristics using functional coverage, then import the data itself and project it onto the coverage model.

For example, imagine you are a Specman user who wants measure & report code development progress.  In Specman you have the reflective facility, which is an API into the type system of the language.  Hence, this capability enables you to collect data on code itself, giving you information about all the types, methods and amount of code developed.  Below is an example of this technique of using functional coverage to model code statistics, where a struct with a coverage group is defined and for all the metrics of interest, cover points are defined.

<'
type pkg_nr: [P1,P2,P3,P4,P5,P6,P7,P8,P9,P10,P11,P12,P13,P14,P15,P16,P17,P18,P19,P20];
struct pkg_info {
   nr: pkg_nr;
   name: string;
   modules: uint;
   lines: uint;
   event cover_me;
   cover cover_me is {
      item nr using per_instance,ignore=nr.as_a(uint) > (rf_manager.get_all_loaded_erm_packages().size()-1);
      item name;
      item modules using ranges = {
         range([1..20],"less than 21");
         range([21..40],"between 21 and 40");
         range([41..1k],"more than 41");
      };
      item lines using ranges = {
         range([1..100],"< 100");
         range([101..1000],"100..1k");
         range([1001..0x2800],"1k..10k");
         range([0x2801..0x7800],"10k..30k");
         range([0x7801..0xc800],"30k..50k");
         range([0xc801..0x1_9000],"50k..100k");
         range([0x1_9001..0x3_2000],"100k..200k");
         range([0x3_2001..0x7_d000],"200k..500k");
         range([0x7_d001..0x10_0000],"500k..1M");
      };
   };
}; // struct pkg_info...
'>

The following code is then used to obtain the data of interest and project it onto the coverage model:

<'
unit pgk_info_cover {
   !info_list: list of pkg_info;

   collect_stats() is {
      var pkgs:= rf_manager.get_all_loaded_erm_packages();
      check pkg_enum_chk that pkgs.size() < 21 else
        dut_error("TestEnv error. Please extend the enum type for packages to match the number of packages you have (default is 20 values)");
      var inf: pkg_info;
      for each (pkg) in pkgs {
         var mods:= pkg.get_modules();
         var lines:= mods.sum(it.get_lines_num());
         inf = new with {
            it.nr = index.as_a(pkg_nr);
            it.name    = pkg.get_name();
            it.modules = mods.size();
            it.lines   = lines;
         };
         info_list.add(inf);
      };
      for each in info_list {
         emit it.cover_me;
      };
   };
   run() is also {
      collect_stats();
   };
};

extend sys {
   pkg_cov: pgk_info_cover is instance;
};

'>

You can then add this coverage to your Enterprise or Desktop Manager vPlan like this:
   . . .
   section "Package_B2" {
  coverage "nr_files" {
  items_pattern : <text>(e,type)pkg_info.cover_me(nr==P2).modules</text>;
 };
 coverage "nr_lines" {
  items_pattern : <text>(e,type)pkg_info.cover_me(nr==P2).lines</text>;
 };
   . . .

The above is just one example on how to import additional metrics, you can also use file I/O (i.e. reading files, parsing for specific information and then mapping it to coverage) in a similar fashion.

Happy coding!

Hannes Froehlich
Solutions Engineer (& Team Specman principal)
Cadence UK

 

Comments(0)

Leave a Comment


Name
E-mail (will not be published)
Comment
 I have read and agree to the Terms of use and Community Guidelines.
Community Guidelines
The Cadence Design Communities support Cadence users and technologists interacting to exchange ideas, news, technical information, and best practices to solve problems and get the most from Cadence technology. The community is open to everyone, and to provide the most value, we require participants to follow our Community Guidelines that facilitate a quality exchange of ideas and information. By accessing, contributing, using or downloading any materials from the site, you agree to be bound by the full Community Guidelines.