Automatic Coverage Closure – my perspective

Recently EDA tools are emerging in the area of “Automatic Coverage Closure” that promise a new level of automation in CDV/MDV/any_other_Buzz_word_Driven_Verification process. A significant name in this arena is nuSym, a relatively new EDA player. There have been few good reviews about them @ Deepchip.com.

http://deepchip.com/items/0479-05.html

http://deepchip.com/items/0473-06.html

http://deepchip.com/items/dvcon07-06.html

And another one @ SiliconIndia:

http://www.siliconindia.com/magazine/articledesc.php?articleid=DPEW289996212

 

And very recently on VerifGuild:

http://www.verificationguild.com/modules.php?name=Forums&file=viewtopic&t=3102

I like Gopi’s post/comment b’cos I have the same opinion about CRV (Constraint Random Verification) – it catches scenarios/bugs that you didn’t envision – either via constraints or coverage (or otherwise). Now if we fool ourself by going behind “only the existing/identified coverage holes” we fall into a trap. This is inline/insync with what Sundaresan Kumbakonam of BRCM ( need his profile? See: http://vlsi-india.org/vsi/activities/dvw05_blr/index.html) shared with me once:

 

Quoting Sundaresan:

 I don’t believe much in the idea of “writing functional coverage model” and then tweaking a constraint here-or-there, or writing a “directed test” for it to fill the hole.

Coming back to my view, I believe some redundancy via randomness/CRV is actually good. In my past verification cycles I have seen design errors due to “repeated patterns” – no big deal, is it?

So where exactly do these ACC tools fit?

Referring back to:

http://www.verificationguild.com/modules.php?name=Forums&file=viewtopic&t=3102

>> whether these tools are only used to reach last few % of coverage goal which is hard to reach ?

I would differ here, they shall be very useful somewhere during the middle phase – neither too early, nor too late. Too early – perhaps we don’t have full RTL and/or functional cov model. Too late – perhaps our focus should be more into “checking” than only coverage (As Nagesh pointed out in VerifGuild). I would like to add that during those last minutes, the coverage shall be taken for “granted” – meaning it is a *must* and not a *nice to have* thing and the focus shall be to look for any failures.

To me a reasonable flow with these ACC tools would be:

• Run with CRV, measure code coverage. Add checkers
• Add functional coverage, use CRV again to hit them using the coverage points as “potential trouble spots” than “actual scenarios” themselves. In few cases where in the scenario description is easy to capture using Functional coverage syntax, this is great. IMHO the existing coverage syntax is little too verbose and unusable to a large extent for solid, easy-to-use coverage specification. Specifically the SV syntax overhead of coverage is just too much for me. IEEE 1647 “e” fairs slightly better but that’s a different story altogether. I’m still on the lookout for a higher level coverage specification language.. (matter for another blog post anyway).
• Once the RTL and the coverage model is reasonably stable, use ACC regularly as a “sanity” test on every interim RTL release. I believe ACC has a HUGE potential here – if we can optimize the tests needed to release interim RTL versions, we are saving quality time and enabling faster turn around.
• Towards the end, enable “plain CRV” (without the ACC bias) and look for “trouble free regression for XX days”.

 

And while speaking to a friend of mine here a while back, he is damn against the idea of using these ACC tools for merely stimulus. He likes the idea of ACC if it can be used to:

• Fill functional cov holes
• Code coverage holes
• Assertion cov misses/holes

A tough ask, but looks like nuSym can handle that – atleast based on the early reviews so far. Also reading their whitepaper on “intelligent verification”, they do a path tracing that enables them to systematically target code coverage without getting into Formal world – cool idea indeed! Kudos to nuSym folks (some of them my ex-colleagues BTW).

And on the application of these ACC tools to the poor, non-CRV/CDV folks – there is light at the end of the tunnel, if you read nuSym’s paper. We at CVC also have ideas on how to use this for a highly configurable IP verification with plain vanilla verilog/task based TBs. We need to prototype it before we can discuss it in detail though.

 

Anyway, good topic for otherwise a downturn mood.

 

More to follow.

Srini

P.S. Sorry for the “random” rambling, after all we are talking of “random verification” :-)

Certificate course on Functional Verification …basics to ASIC verification using SystemVerilog

CVC is about to launch a 10-day certificate course on Functional Verification covering SystemVerilog in depth. Broadly it covers the following topics:

• Comprehensive introduction to Functional Verification (CFV)
• SystemVerilog basics (SVB)
• SystemVerilog Assertion (SVA)
• Verification Using SystemVerilog (VSV)
• Verification Methodology (VM)

Duration

Here is a detailed breakdown of the course with duration. Note that we have several “mini projects” tightly embedded in the course that helps in mastering topics learned so far in the course. This is on top of the regular labs that are part of the training. The detailed breakup of topics and labs is covered in next sections of this proposal.

Topic	Duration
Comprehensive Functional Verification	1.5 days
Mini Project I	0.5 day
SystemVerilog Basics	0.5 day
SystemVerilog Assertions	1.5 days
Mini Project II	0.5 day
Verification using SystemVerilog	2.0 days
Mini Project III	0.5 day
Verification Methodology	2.0 days
Project IV	1.0 day

Schedule

Tentative: Feb 09-Mar09

Contact

Send an email to: cvc.training@gmail.com and/or training@noveldv.com for more details, cost etc. Or call us at: +91-9916176014

When will SV Interface be really useful?

The idea of adding interface construct to SV language has proven to be a short sighted one with all its ugly ramifications for the RTL side (refer to good papers on this from Jonathan @Douolos if you need proof). Add to it the fact that not all RTL synthesis (+ FPGA), Linters, Equiv. checkers fully supporting it yet!

Atleast on Verification front it has been proving good. However a significant drop has been the lack of proper debug support for it. I wish EDA takes debug seriously. It affects productivity so much that any language level gain we get is nullified with lack or weak support for these new constructs.

For instance, look at SV Interface as a simple "wire bundle" - do we have debuggers handle it at that level of abstraction?

Luckily Verdi seems to be doing it (leading the way as ever before), see:

http://newsletter.springsoft.com/.docs/pg/10768

Happy debugging!

Ajeetha, CVC
www.noveldv.com

Excellent case study on automatic Coverage closure – nuSym & QCOM

An absolute *must read* for all those CDV/CRV fans (FWIW: CDV – Coverage Driven Verification, CRV – Constrained Random Verification):

 

http://www.deepchip.com/items/0479-05.html

 

A live case study from Jim @QCOM. It has good details about the setup, work done and results. Looks like nuSym does deliver the kind of promises/claims it makes, good going indeed! Based on the last 2 such results (both @deepchip.com) I have few observations:

 

1. Both of them were using Vera against SystemVerilog. While the technology shall be language independent, it will be good to get a SV case study out as well

2. I’m not very clear why and how nuSym can replace a core “simulator” – there are just lot more things in a “simulator” than just “coverage closure/intelligence” – what about debug, stability, memory footprint, gate level/ASIC sign off, dumping, Debussy like integration etc etc.? I fully appreciate the smartness in random generation – it is time EDA folks did that in so called modern “Verification platforms”. But I fail to see how a point tool like nuSym can “replace” a simulator, instead it shall augment it and bring the bigwig EDA vendor pricing down to a reasonable bargain :-)

 

More notes as we read/re-read that article.

 

Anyway thanks Jim for sharing those wonderful details!

 

Cheers

Srini

CTO @CVC www.noveldv.com

SVA challenges in creating, debugging and verifying assertions

During our SVA class today there were frequent questions/comments on:

 

• How to write assertions easily without becoming a language guru?
• How to ensure that the assertions we write are correct to start with? (Not syntax wise, rather functionally)
• How to visualize assertions/attempts/threads easily?
• Can we create assertions automatically from a timing diagram/dump file?
• How to debug assertions? What sort of automation is available?
• Given a dump file, can we explore a set of assertions about that design (without having to rerun the simulations)?
• Can we verify assertions in isolation – i.e. even before RTL and/or TB is ready?

Sure a boatload of questions, some answers are available some are not as of today. CVC will address some of these in a seminar in one of the coming weeks. (Stay tuned to this site for that news).

Here are some answers:

Q: How to write assertions easily without becoming a language guru?

• Leverage on assertion libraries such as OVL, VMM SVA lib, QVL, IAL etc.

Q: How to ensure that the assertions we write are correct to start with? (Not syntax wise, rather functionally)

• Not an easy thing, but again use pre-verified assertion lib elements (see prev Q)

Q: How to visualize assertions/attempts/threads easily?

• Know your tools: Springsoft (formerly Novas) has a great product called Verdi that can present a “Temporal Flow View” and thread view. It is so amazing and intuitive that you will stay locked with it for long long time to come, really speaking. The idea of temporal annotation is not new, we spoke about it in our PSL book, see below a snapshot:

 

 

The core idea is to annotate the values of signals at “appropriate time” and not just based on current time (the latter is what most of debuggers do, except for Verdi AFAIK).

 

Consider a code like:

 

mul_attempts : assert property (@posedge clk) start |=> s1;

sequence s1;

a ##1 b ##2 c;

endsequence

 

Below is a screenshot of how Verdi can display it.

 

 

2 key/novel ideas here to appreciate:

 

1. The threads are nicely displayed with “horizontal lines” – this is exactly how our PPT in training explains threads BTW!

2. The Temporal annotation of the sequence/property with values & time stamps. For the failure at 350 ns (assume 20 ns clock period), it shows value of: (a ##1 b ##2 c;)

• “c” @350ns
• “b” @310 ns
• “a” @290
• “start” @270ns

This is simply superb, hats off to Verdi!

Q: How to debug assertions? What sort of automation is available?

• Refer to prev Q, almost every vendor provides some automation.

Q: Can we create assertions automatically from a timing diagram/dump file?

• Yes, there have been few attempts at this. OneSpin claims this feature and also a new start-up http://solidoaktech.com/TD2Assert.html We will investigate more and update here soon.

 

Q: Given a dump file, can we explore a set of assertions about that design (without having to rerun the simulations)?

• VCS can do this
• Springsoft/Verdi can do this
• Veritools can do this too!

Q: Can we verify assertions in isolation – i.e. even before RTL and/or TB is ready?

• Strictly speaking this is ideal job for formal verification tools. We believe some tools such as Magellan (Synopsys) already do this. We will update more here, stay tuned (for the 3rd time in this post…)

Week long fest on Verification Using SystemVerilog - Bangalore

 

Quick facts
When: Feb 2 to Feb 6 2009

Cost: Rs. 5000 /- per day

Contact: cvc.training @ gmail.com, +91-9916176014, +91-80-42134156

What’s SystemVerilog?
IEEE 1800, SystemVerilog is the de-facto language for Digital system Verification (and Design). Almost every ASIC team is either using it or plan on using it in the next project! It is a major extension to Verilog-2001, adding significant new features to Verilog for verification and design. Enhancements range from simple enhancements to existing constructs, addition of new language constructs to the inclusion of complete Object-Oriented paradigm features.

What’s a week long fest?
A week long fest on SystemVerilog for Verification is aimed at introducing SystemVerilog in its full capacity covering basics, assertions, testbench features and ending with methodology. At the end of this fest the essential features of SystemVerilog shall be covered and will enable you develop complex testbenches using advanced techniques such as OOP, Constrained Random Verification and Coverage Driven Verification.  It is aimed at novice SV users and hence language is dealt with a target DUT in picture. The goal is to put SV to use for real life verification than understand the nitty gritties of the language from semantic/gotchas perspective.

Who should attend?
Practicing Design and Verification engineers with tight project schedules are ideal attendees. DV managers will equally find it useful as they can grasp the complexity of SV. Though it is strongly recommended to attend the whole 5 day fest, some may choose just the assertions/testbench/methodology and be present in those days accordingly. Please call use for more details.

Tools used

• Questa/Modelsim (Mentor)
• VCS (Synopsys)
• Riviera (Aldec) - optional

What’s the cost?
The basic cost of this course is Rs. 5,000 /- + ST (12.36 %) per day per attendee.

Terms & Conditions
· In general we require that the fee is paid in 100% prior to the start of the training.
· For large corporate with more number of attendees to account for their internal process we do allow an exception to the above rule; however we charge an additional 25% of the training cost per attendee in such cases. In case the fee is paid after the training, the payment should be made within 1 week after the training is delivered. Any additional delay shall be charged at 10% every additional day.
· Any "offer" price mentioned in the course announcement is applicable only for individual attendees and not for corporate.

Cancellation Policy
Course tuition is fully refundable up to one week before the class starts. Cancellations within a week (2-7 days) of the class start date will incur a 50% cancellation fee. Those who cancel fewer than 2 days prior to the class will be billed for the full amount of the tuition. A no-show will be treated as cancellation and no refund shall be given. For genuine cases of absence, we can provide a training token that the trainee can avail in one of the future training classes subject to space availability.

How do I register for a class?
To attend this class, confirm your registration by sending an email to cvc.training @ gmail.com. +91-9916176014, +91-80-42134156

Please include the following details in your email:
Name:

Company Name:

Official Email ID:

Contact Number:

Trainer Profile

Srinivasan Venkataramanan, CTO

http://www.linkedin.com/in/svenka3

• Over 12 years of experience in VLSI Design & Verification
• Co-authored leading books in the Verification domain.
• Worked at Philips, Intel, Synopsys in various capacities.
• Presented papers, tutorials in various conferences, publications and avenues.
• Conducted workshops and trainings on PSL, SVA, SV, VMM, E, ABV, CDV and OOP for Verification
• Holds M.Tech in VLSI Design from prestigious IIT, Delhi.

Ajeetha Kumari, CEO & MD
· Has 8+ years of experience in Verification
· Co-authored leading books in the Verification domain.
· Presented papers, tutorials in various conferences, publications andavenues.
· Worked with all leading edge simulators and formal verification(Model Checking) tools.
· Conducted workshops and trainings on PSL, SVA, SV, VMM, E, ABV, CDVand OOP for Verification
· Holds M.S.E.E. from prestigious IIT, Madras.