Automatic generation of checkers & coverage model – A NextOp 101

 

Earlier this week John’s DeepChip ran a user survey asking for “edgy” questions for DAC-12 “Troublemaker Panel”. Here is what those came out for NextOp were: http://www.deepchip.com/items/0504-05.html

>> Yunshan, what does NextOp do and why should users buy your tool?

I am amazed at how “basic” some of these “queries” are – aren’t they supposed to be “edgy”? Sure John is doing a great job in publishing “AS-IS”, so can’t blame him for this, it is rather the typical limitation of small start-up, especially in EDA being unable to broadcast its value to masses. Here is my attempt, being a partner, promoter of this technology and SystemVerilog in general.

Assume that you are tasked with a I2C IP/sub-system verification (it isn’t that uncommon, is it?). Consider an AMBA based SoC in which this I2C is being integrated, the other side of this IP is typically APB.

Your company has tight timelines for this and has provided you VIPs a la modern day UVC (UVM Verification Component). So your job is to simply hook-up those UVCs as shown below:

 

Assuming the UVCs live up to their promise of plug-n-play by your VIP vendor, this would be a 1 or 2 days job isn’t it? Then add few more days to tighten the “scenarios”, then you are done in a week, right?

So a typical I2C verification project should be atmost 1 week project, Huh? Is life really that simple? Of-course not! – this is where NextOp plugs-in. UVM is great, and hopefully the maturing VIP industry around it can indeed provide you solid plug-n-play-able VIPs for you. This does automate majority of “stimulus” creation and some of “protocol checkers” and “functional coverage”. But what about “your design specific” checkers & coverage? Isn’t that the MOST important and hence is the reason why you are tasked with this project to start with?

So how do automate the “design specific” checkers & coverage? This is how – a NextOp 101 starts here.

 

 

You run the UVCs + RTL design as-is (after that 1 week of your project start, let’s say). Include 2 simple PLI calls:

$himafile & $himavars

This flow would eventually create a set of “properties” or “observations” that your TB + RTL is showing up during the regressions as a simple, plain English TEXT file. No, it is not a syntatically “overloaded/crowded” SVA/PSL code for your “kind review”, rather simple format such as (BTW, the below screenshot is from a  different, DDR design, just got lazy enough not to do that on that specific design):

 

Now you sit with your designers and classify them as “assertions” and “coverage holes”. Typically the designer spends 2-5 minutes per-property to decide this classification. Then you run another “utility” called “hima” that spits out SVA/PSL/OVL/Verilog assertions for you to add to your UVM bench during next random regression.

It is quite common that users identify bugs/coverage holes during the review itself than waiting for next runs!

Now quoting a reader’s question from that Jonh’s “edgy questions” (I do agree this is an “edgy question”)

 Why should we pay extra for BugScope when we can get SNPS/CDNS/MENT
  formal verification tools for "free" (or at a heavy discount) as
  part of a bundled deal?

 

Hopefully this blog entry answered it atleast partially – it is NOT available as part of any other EDA vendor as of today!

Yes, you do pay extra – bu then YOU-GET-WHAT-YOU-PAY-FOR :-)

Enjoy the ABV with NextOp!

Technorati Tags: SystemVerilog,EDA,NextOp,Assertions,checkers,coverage

Automatic Coverage Closure – my perspective

Recently EDA tools are emerging in the area of “Automatic Coverage Closure” that promise a new level of automation in CDV/MDV/any_other_Buzz_word_Driven_Verification process. A significant name in this arena is nuSym, a relatively new EDA player. There have been few good reviews about them @ Deepchip.com.

http://deepchip.com/items/0479-05.html

http://deepchip.com/items/0473-06.html

http://deepchip.com/items/dvcon07-06.html

And another one @ SiliconIndia:

http://www.siliconindia.com/magazine/articledesc.php?articleid=DPEW289996212

 

And very recently on VerifGuild:

http://www.verificationguild.com/modules.php?name=Forums&file=viewtopic&t=3102

I like Gopi’s post/comment b’cos I have the same opinion about CRV (Constraint Random Verification) – it catches scenarios/bugs that you didn’t envision – either via constraints or coverage (or otherwise). Now if we fool ourself by going behind “only the existing/identified coverage holes” we fall into a trap. This is inline/insync with what Sundaresan Kumbakonam of BRCM ( need his profile? See: http://vlsi-india.org/vsi/activities/dvw05_blr/index.html) shared with me once:

 

Quoting Sundaresan:

 I don’t believe much in the idea of “writing functional coverage model” and then tweaking a constraint here-or-there, or writing a “directed test” for it to fill the hole.

Coming back to my view, I believe some redundancy via randomness/CRV is actually good. In my past verification cycles I have seen design errors due to “repeated patterns” – no big deal, is it?

So where exactly do these ACC tools fit?

Referring back to:

http://www.verificationguild.com/modules.php?name=Forums&file=viewtopic&t=3102

>> whether these tools are only used to reach last few % of coverage goal which is hard to reach ?

I would differ here, they shall be very useful somewhere during the middle phase – neither too early, nor too late. Too early – perhaps we don’t have full RTL and/or functional cov model. Too late – perhaps our focus should be more into “checking” than only coverage (As Nagesh pointed out in VerifGuild). I would like to add that during those last minutes, the coverage shall be taken for “granted” – meaning it is a *must* and not a *nice to have* thing and the focus shall be to look for any failures.

To me a reasonable flow with these ACC tools would be:

• Run with CRV, measure code coverage. Add checkers
• Add functional coverage, use CRV again to hit them using the coverage points as “potential trouble spots” than “actual scenarios” themselves. In few cases where in the scenario description is easy to capture using Functional coverage syntax, this is great. IMHO the existing coverage syntax is little too verbose and unusable to a large extent for solid, easy-to-use coverage specification. Specifically the SV syntax overhead of coverage is just too much for me. IEEE 1647 “e” fairs slightly better but that’s a different story altogether. I’m still on the lookout for a higher level coverage specification language.. (matter for another blog post anyway).
• Once the RTL and the coverage model is reasonably stable, use ACC regularly as a “sanity” test on every interim RTL release. I believe ACC has a HUGE potential here – if we can optimize the tests needed to release interim RTL versions, we are saving quality time and enabling faster turn around.
• Towards the end, enable “plain CRV” (without the ACC bias) and look for “trouble free regression for XX days”.

 

And while speaking to a friend of mine here a while back, he is damn against the idea of using these ACC tools for merely stimulus. He likes the idea of ACC if it can be used to:

• Fill functional cov holes
• Code coverage holes
• Assertion cov misses/holes

A tough ask, but looks like nuSym can handle that – atleast based on the early reviews so far. Also reading their whitepaper on “intelligent verification”, they do a path tracing that enables them to systematically target code coverage without getting into Formal world – cool idea indeed! Kudos to nuSym folks (some of them my ex-colleagues BTW).

And on the application of these ACC tools to the poor, non-CRV/CDV folks – there is light at the end of the tunnel, if you read nuSym’s paper. We at CVC also have ideas on how to use this for a highly configurable IP verification with plain vanilla verilog/task based TBs. We need to prototype it before we can discuss it in detail though.

 

Anyway, good topic for otherwise a downturn mood.

 

More to follow.

Srini

P.S. Sorry for the “random” rambling, after all we are talking of “random verification” :-)