CCD, My read of Certess technology and positioning

 

With due respect to the technology behind Certess’s tool I have some discomfort with the way it is being positioned – atleast in the below article:

http://www.edadesignline.com/howto/215600203;jsessionid=TP12OA3IF1X3UQSNDLOSKHSCJUNN2JVN?pgno=2

Before I talk about my discomfort, let me state the positives: Not very often do we get to read such well written, all encompassing technical article, Kudo’s to Mark Hampton – he touches on every aspect of functional verification in this article, not so common in an EDA product “promotional” article – to which this article may be characterized to (unfortunately IMHO). Having said that, I personally believe Certess should position the technology “along with” existing ones than challenging/trying to replace time tested/well adopted methodologies such as code cov, functional cov etc. Not that I differ from his views on the shortcomings of these technologies, rather going by what Pradip Thakcker said in DVM 08 (http://vlsi-india.org/vsi/activities/2008/dvm-blr-apr08/program.html)

“Code coverage and functional coverage are useful techniques with their own strengths and weaknesses. Rather than worrying about their weaknesses, focus on the positives and use them today”..Pradip, during his “Holistic Verification: Myth or The Magic Bullet?”

I will be very glad if Certess focuses on their real strength of exposing lack of checkers in a Verification environment than trying to “eat” into the well established market of Code/Func coverage tools. Another rationale: Both the cov and qualification is compute intensive and given the amount of EDA investment that has gone into stabilizing and optimizing these features, it will be irrational to try and replace them with “functional qualification” (No offense meant, I have great respect for Mark – given his excellent article and ofcourse the product). With SpringSoft acquiring Certess hopefully their customer base/reach increases and that will throw up more success stories in the coming months/quarters. So good times ahead!

ITG, C2D & ACC - Emerging Verification technologies

Well, it is not the overly hyped *V here - such as CRV, CDV, ABV - we at CVC (www.noveldv.com) consider them as yesterday-ones for the sake of giving room to next generation ones such as:

• ACC - Automatic Coverage Closure 
• ITG - Intelligent Test Generation (such as Graph based)
• CCD - Covered & Checked implies Done (such as Certess/SpringSoft)

Out of this let me spend more time on the last two as the ACC is already been discussed for a while now (atleast more than the other two).

ITG - Intelligent Test Generation (such as Graph based)

ITG - is still in its early days. Two tools seem to be addressing this as of today:

1. Infact  from Mentor is one big name.
2. The other one that is very promising is: Breker Systems with a very high profile team behind it. These folks know what they are talking about - with their CTO holding "Adnan holds 15 patents in test case generation and synthesis.".

We at CVC are yet to get our hands dirty with these tools, but certainly worth watching indeed! From our early analysis this technology will assist more and more system level tests being easily captured by raising the level of abstraction of testcase specification. This will be fun indeed!

CCD - Covered & Checked implies Done 

Coming to the other category: CCD (yet to find a better name) – this is a topic that has been haunting us for atleast a decade now. Ever since I started using Functional coverage (early 2000), we always had this problem of “I got it covered, but did I get it checked too?”. During an Ethernet monster switch/router verification at Intel we hit this problem atleast half-a-dozen times and those corridor discussions still ring in my ears. The Design (read it as RTL) manager (Sutapa Chandra) made fun of us asking “are we taping out RTL or testbench” as we seem to be finding lack of checkers every now and then. Most of these situations are the case of bugs went undetected at block/cluster level and later get got (luckily) at full chip level – then we do a rigorous review of our block level env, and find that we indeed had coverage points for those scenarios, just that we didn’t have enough checkers! Shame, but true. A technology such as Certess’s Testbench Qualification was what was indeed needed! A very detailed read of Certess technology is at: http://www.edadesignline.com/howto/215600203;jsessionid=TP12OA3IF1X3UQSNDLOSKHSCJUNN2JVN?pgno=2

OVM rule checker for free!

A process/methodology is only as good as its adoption/compliance. And in Verification since there are so many different ways of “getting it done” it is hard to get sync across teams/groups/projects/companies etc. This is where the methodology comes to play (be it VMM,OVM<eRM etc.). But again how am I sure that I follow all the guidelines?

 

Consider this – we got an email from a user as recently as last  week on our VMM book code (www.systemverilog.us) about not adding “is_valid” method in data model. The book was published back in 2006 I believe. That just goes to say that a “compliance checker” is a very handy tool to create highly reusable Verification code. But who will check for compliance?

 

Thankfully, here is an answer for OVM and that too for no cost!

http://www.veriez.com/dvcon_2009_pr.htm<a href="http://www.veriez.com/dvcon_2009_pr.htm"></a> This is indeed an interesting development and a welcome one during tight economical situations. Make sure you download and use them!

 

Cheers,

Ajeetha www.noveldv.com

Automatic Coverage Closure – my perspective

Recently EDA tools are emerging in the area of “Automatic Coverage Closure” that promise a new level of automation in CDV/MDV/any_other_Buzz_word_Driven_Verification process. A significant name in this arena is nuSym, a relatively new EDA player. There have been few good reviews about them @ Deepchip.com.

http://deepchip.com/items/0479-05.html

http://deepchip.com/items/0473-06.html

http://deepchip.com/items/dvcon07-06.html

And another one @ SiliconIndia:

http://www.siliconindia.com/magazine/articledesc.php?articleid=DPEW289996212

 

And very recently on VerifGuild:

http://www.verificationguild.com/modules.php?name=Forums&file=viewtopic&t=3102

I like Gopi’s post/comment b’cos I have the same opinion about CRV (Constraint Random Verification) – it catches scenarios/bugs that you didn’t envision – either via constraints or coverage (or otherwise). Now if we fool ourself by going behind “only the existing/identified coverage holes” we fall into a trap. This is inline/insync with what Sundaresan Kumbakonam of BRCM ( need his profile? See: http://vlsi-india.org/vsi/activities/dvw05_blr/index.html) shared with me once:

 

Quoting Sundaresan:

 I don’t believe much in the idea of “writing functional coverage model” and then tweaking a constraint here-or-there, or writing a “directed test” for it to fill the hole.

Coming back to my view, I believe some redundancy via randomness/CRV is actually good. In my past verification cycles I have seen design errors due to “repeated patterns” – no big deal, is it?

So where exactly do these ACC tools fit?

Referring back to:

http://www.verificationguild.com/modules.php?name=Forums&file=viewtopic&t=3102

>> whether these tools are only used to reach last few % of coverage goal which is hard to reach ?

I would differ here, they shall be very useful somewhere during the middle phase – neither too early, nor too late. Too early – perhaps we don’t have full RTL and/or functional cov model. Too late – perhaps our focus should be more into “checking” than only coverage (As Nagesh pointed out in VerifGuild). I would like to add that during those last minutes, the coverage shall be taken for “granted” – meaning it is a *must* and not a *nice to have* thing and the focus shall be to look for any failures.

To me a reasonable flow with these ACC tools would be:

• Run with CRV, measure code coverage. Add checkers
• Add functional coverage, use CRV again to hit them using the coverage points as “potential trouble spots” than “actual scenarios” themselves. In few cases where in the scenario description is easy to capture using Functional coverage syntax, this is great. IMHO the existing coverage syntax is little too verbose and unusable to a large extent for solid, easy-to-use coverage specification. Specifically the SV syntax overhead of coverage is just too much for me. IEEE 1647 “e” fairs slightly better but that’s a different story altogether. I’m still on the lookout for a higher level coverage specification language.. (matter for another blog post anyway).
• Once the RTL and the coverage model is reasonably stable, use ACC regularly as a “sanity” test on every interim RTL release. I believe ACC has a HUGE potential here – if we can optimize the tests needed to release interim RTL versions, we are saving quality time and enabling faster turn around.
• Towards the end, enable “plain CRV” (without the ACC bias) and look for “trouble free regression for XX days”.

 

And while speaking to a friend of mine here a while back, he is damn against the idea of using these ACC tools for merely stimulus. He likes the idea of ACC if it can be used to:

• Fill functional cov holes
• Code coverage holes
• Assertion cov misses/holes

A tough ask, but looks like nuSym can handle that – atleast based on the early reviews so far. Also reading their whitepaper on “intelligent verification”, they do a path tracing that enables them to systematically target code coverage without getting into Formal world – cool idea indeed! Kudos to nuSym folks (some of them my ex-colleagues BTW).

And on the application of these ACC tools to the poor, non-CRV/CDV folks – there is light at the end of the tunnel, if you read nuSym’s paper. We at CVC also have ideas on how to use this for a highly configurable IP verification with plain vanilla verilog/task based TBs. We need to prototype it before we can discuss it in detail though.

 

Anyway, good topic for otherwise a downturn mood.

 

More to follow.

Srini

P.S. Sorry for the “random” rambling, after all we are talking of “random verification” :-)

Certificate course on Functional Verification …basics to ASIC verification using SystemVerilog

CVC is about to launch a 10-day certificate course on Functional Verification covering SystemVerilog in depth. Broadly it covers the following topics:

• Comprehensive introduction to Functional Verification (CFV)
• SystemVerilog basics (SVB)
• SystemVerilog Assertion (SVA)
• Verification Using SystemVerilog (VSV)
• Verification Methodology (VM)

Duration

Here is a detailed breakdown of the course with duration. Note that we have several “mini projects” tightly embedded in the course that helps in mastering topics learned so far in the course. This is on top of the regular labs that are part of the training. The detailed breakup of topics and labs is covered in next sections of this proposal.

Topic	Duration
Comprehensive Functional Verification	1.5 days
Mini Project I	0.5 day
SystemVerilog Basics	0.5 day
SystemVerilog Assertions	1.5 days
Mini Project II	0.5 day
Verification using SystemVerilog	2.0 days
Mini Project III	0.5 day
Verification Methodology	2.0 days
Project IV	1.0 day

Schedule

Tentative: Feb 09-Mar09

Contact

Send an email to: cvc.training@gmail.com and/or training@noveldv.com for more details, cost etc. Or call us at: +91-9916176014

When will SV Interface be really useful?

The idea of adding interface construct to SV language has proven to be a short sighted one with all its ugly ramifications for the RTL side (refer to good papers on this from Jonathan @Douolos if you need proof). Add to it the fact that not all RTL synthesis (+ FPGA), Linters, Equiv. checkers fully supporting it yet!

Atleast on Verification front it has been proving good. However a significant drop has been the lack of proper debug support for it. I wish EDA takes debug seriously. It affects productivity so much that any language level gain we get is nullified with lack or weak support for these new constructs.

For instance, look at SV Interface as a simple "wire bundle" - do we have debuggers handle it at that level of abstraction?

Luckily Verdi seems to be doing it (leading the way as ever before), see:

http://newsletter.springsoft.com/.docs/pg/10768

Happy debugging!

Ajeetha, CVC
www.noveldv.com

Excellent case study on automatic Coverage closure – nuSym & QCOM

An absolute *must read* for all those CDV/CRV fans (FWIW: CDV – Coverage Driven Verification, CRV – Constrained Random Verification):

 

http://www.deepchip.com/items/0479-05.html

 

A live case study from Jim @QCOM. It has good details about the setup, work done and results. Looks like nuSym does deliver the kind of promises/claims it makes, good going indeed! Based on the last 2 such results (both @deepchip.com) I have few observations:

 

1. Both of them were using Vera against SystemVerilog. While the technology shall be language independent, it will be good to get a SV case study out as well

2. I’m not very clear why and how nuSym can replace a core “simulator” – there are just lot more things in a “simulator” than just “coverage closure/intelligence” – what about debug, stability, memory footprint, gate level/ASIC sign off, dumping, Debussy like integration etc etc.? I fully appreciate the smartness in random generation – it is time EDA folks did that in so called modern “Verification platforms”. But I fail to see how a point tool like nuSym can “replace” a simulator, instead it shall augment it and bring the bigwig EDA vendor pricing down to a reasonable bargain :-)

 

More notes as we read/re-read that article.

 

Anyway thanks Jim for sharing those wonderful details!

 

Cheers

Srini

CTO @CVC www.noveldv.com