Home > Community > Forums > Functional Verification > Tracking regression results with an SQLITE database

Email

* Required Fields

Recipients email * (separate multiple addresses with commas)

Your name *

Your email *

Message *

Contact Us

* Required Fields
First Name *

Last Name *

Email *

Company / Institution *

Comments: *

 Tracking regression results with an SQLITE database 

Last post Thu, Jan 17 2013 2:41 AM by StephenH. 4 replies.
Started by andymont 15 Jan 2013 08:11 AM. Topic has 4 replies and 1515 views
Page 1 of 1 (5 items)
Sort Posts:
  • Tue, Jan 15 2013 8:11 AM

    • andymont
    • Not Ranked
    • Joined on Tue, Sep 4 2012
    • Posts 4
    • Points 50
    Tracking regression results with an SQLITE database Reply

    Hi , 

    I would like to keep track of the verification status of my project (verilog/systemverilog) coverage and pass/fail status of tests during the life of the project. Ideally I want to create an SQLITE database to keep track of this since SQLITE is stored in a file which can be archived with the project after tapeout.

    I am already writing my own solution which involves parsing the output vsof from a regression and then storing the information I need, however this seems to me to be something which may already be possible and there does seem to be an option in the emanager configuration to use SQL but there is no information in any of the docs which explains its use.

    Does anyone have any ideas ?

    regards

    Andy 

    • Post Points: 20
  • Tue, Jan 15 2013 9:35 AM

    • StephenH
    • Top 25 Contributor
    • Joined on Tue, Sep 2 2008
    • Bristol, Avon
    • Posts 268
    • Points 4,310
    Re: Tracking regression results with an SQLITE database Reply

    Hello Andy!

    I would generally advise against parsing the VSOF directly as its format does occasionally change slightly and I have seen users flows break in this circumstances. Usually at the point where the author of the parser script has moved on and nobody knows what it was meant to do... ;-)

    You could use the API provided by vManager to load up the VSOF(s) and query the run status (pass/fail/incomplete) and get the associated failure information. I think you'll find this a lot easier than parsing the VSOF anyway, even if the VSOF format remains stable. If you wish, I can give you some help to get started with the vMananger batch API, or you can dive right in yourself, just bring up cdnshelp and search for vm_manager.read_session(), vm_vsof.get_runs(), vm_run.get_status(). These should get you close enough that you'll figure it out easily enough.

    As regards the SQL support in vManager, this isn't meant for general consumption; it is a legacy thing which isn't under development.

     

    Steve Hobbs / Applications Engineer / Cadence Functional Verification
    Filed under:
    • Post Points: 20
  • Tue, Jan 15 2013 1:57 PM

    • andymont
    • Not Ranked
    • Joined on Tue, Sep 4 2012
    • Posts 4
    • Points 50
    Re: Tracking regression results with an SQLITE database Reply

    Hi Steve, 

     Thanks for the quick reply. I obviously don't want to have to write a parser if I don't need to , is there a good way to export the information from the vmanager API ?  What I don't want to have to do is keep vsofs from every regression or indeed to keep regression data from every run. If I can export the information I need then using the built in API is definitely the way I would like to go.

    cheers

    Andy 

    • Post Points: 20
  • Wed, Jan 16 2013 1:14 AM

    • StephenH
    • Top 25 Contributor
    • Joined on Tue, Sep 2 2008
    • Bristol, Avon
    • Posts 268
    • Points 4,310
    Re: Tracking regression results with an SQLITE database Reply
    Andy, yes it is possible to use the API for that.

    Drop me an email and we can arrange to have a chat about it.
    Steve Hobbs / Applications Engineer / Cadence Functional Verification
    • Post Points: 5
  • Thu, Jan 17 2013 2:41 AM

    • StephenH
    • Top 25 Contributor
    • Joined on Tue, Sep 2 2008
    • Bristol, Avon
    • Posts 268
    • Points 4,310
    Re: Tracking regression results with an SQLITE database Reply

    I sent Andy some example code, which I'm copying here in case it's of help to any other forum members.

    // Usage example:
    // vmanager -batch -c "load dump_results_csv; analyze_runs -vsof my_session/my_session.vsof; dump_results_csv()"
    <'
    extend global {
      dump_results_csv ( result_file_name:string=NULL ) is {
        if result_file_name == NULL { result_file_name = "results.csv"; };
    
        // Get all loaded sessions (there can be more than one)
        var all_sessions := vm_manager.get_all_sessions();
        // Get an analysis "context"
        var ctx : vm_context = vm_manager.create_context(all_sessions.as_a(list of vm_attribute_container), "Default");
        // Get all the runs from that context
        var all_runs := ctx.get_runs();
    
        // Create the output file and header
        var result_file := files.open(result_file_name,"w","Text file");
        files.write(result_file,"Test, Seed, Status, First Failure");
    
        // Get a handle to the sv_seed attribute type
        var attr_seed  := vm_manager.get_attribute_by_name("sv_seed");
        // Get a handle to the test_name attribute type
        var attr_title := vm_manager.get_attribute_by_name("test_name");
    
        // Iterate through all the runs
        for each (run) in all_runs {
          var line :string;
          line = append(run.get_attribute_value_or_default(attr_title), ", ");
          line = append(line, run.get_attribute_value_or_default(attr_seed), ", ");
          line = append(line, run.get_status(), ", ");
          if (run.get_status_enum() == failed) {
            // var all_failures := run.get_failures(); // Alternative if you want all failures
            var first_failure := run.get_first_failure();
            line = append(line, first_failure.get_value("description"));
          } else {
            line = append(line, "n/a");
          };
          files.write(result_file,line);
        };
        files.close(result_file);
      };
    }; 
    

    '> 

    Steve Hobbs / Applications Engineer / Cadence Functional Verification
    Filed under:
    • Post Points: 5
Page 1 of 1 (5 items)
Sort Posts:
Started by andymont at 15 Jan 2013 08:11 AM. Topic has 4 replies.