Wednesday, March 17, 2010

A modern approach to SoC level verification

 

           Verifying SoC is fun and tedious. Especially with several buzz words around the corner, it is quite easy to get lost in maze of buzz-words and miss the goal. At the end one may feel that the plain old wisdom of whiteboard based testcase review/plan is/was lot more controllable & observable. We did that back in 2000 @ Realchip communications and yes it worked really well. But with shrinking times and mounting complexity is that really fast enough? Before I hear constrained-random, blink for a while – how much random do you want your end-to-end data flow in-and-out of ASIC/SoC to be?

We at CVC (www.cvcblr.com) take pride in partnering with all major EDA vendors (http://www.cvcblr.com/partners) – big & small to look for best possible solution for different problems than suggesting “one-size-fit-all” like solution.

Here is a relevant thread @Vguild: http://www.verificationguild.com/modules.php?name=Forums&file=viewtopic&p=17615#17615

I am due to start work on an ASIC, and am wondering about a suitable verification strategy. The ASIC consists of a data path, with continuous data input from ADCs and continuous output to DACs, and a couple of embedded processors utilising external flash and SRAM.

So the interfaces to the ASIC are pretty much:
(1) parallel data bus in
(2) parallel data bus out
(3) external memory interface for CPUs

And here is our own experience/view of some emerging approach to this problem – we don’t claim to have solved it completely, but seem to be making good progress towards a methodical and controllable (yet scalable) manner.

 

Hi Siskin,
Good question/topic. While the value of OVM/VMM is very profound at block levels, their usage at SoC level wherein end-to-end data flow is being checked is not very well reported (yet) in literature. Needless to say they are far better than inventing your own. Especially if you have block-to-system reuse of these VIP components they definitely come very handy. The virtual sequences/multi-stream scenarios do assist but IMHO they come with heavy workouts. Instead what we promote to our customers here and have been proto-typing with at CVC is the solution from Breker, it is called Trek. It can work on top of any existing TB - Verilog/VHDL/TCL/VMM/OVM you name it.
Idea is to reuse the block level components to do what they do best and build tests at a higher level - in this case using graphs, nodes etc. I tend to like this as I used to like Petri nets during my post-graduation days (though didn't followup on my interest afterwards).
My first impression was to use Trek simply as a testcase creation engine but slowly I'm getting convinced it is useful as "checker" as well - especially the end-to-end checks.
You are absolutely right - use assertions in IP interface levels and use some sort of higher level stimulus. Where I see Trek useful in SoC verification is the ability to describe your "flow of data through SoC" as a graph and let the tool generate tests for you. I even jokingly say one can use a palmtop/PDA to draw these graphs during travel, convert them to Trek graph (somehow, didn't chase that dream yet) and have tests ready while I'm on travel - flight/train/bus whatever be it! On a serious note, this is quite similar to how we used to discuss our testplans on a whiteboard during our Realchip (a communication startup in 2000-2001) days, now becoming "executable" Smile
See ST's usage of Trek @
http://www10.edacafe.com/nbc/articles/view_article.php?articleid=787856
Feel free to contact me offline if you need further assistance on Trek. We have our 2nd successful project finishing on using Trek, though these are small/medium scale ones.
My 2 cents!
Srini
www.cvcblr.com
_________________
Srinivasan Venkataramanan
Chief Technology Officer, CVC www.cvcblr.com
A Pragmatic Approach to VMM Adoption
SystemVerilog Assertions Handbook
Using PSL/SUGAR 2nd Edition.
Contributor: The functional verification of electronic systems

Saturday, March 13, 2010

NextOp’s Assertion Synthesis – expanding ABV applications?

 

In case you missed it, read a user report on NextOp’s technology at: http://www.deepchip.com/items/0484-01.html 

In next couple of blog entries, I will share my reading, reflections on this detailed report.

To start with, this technology seems to address some of the “points to ponder” being discussed at: http://www.cvcblr.com/blog/?p=146 

As there is no whitepaper/material available on this technology I base my reflections solely on the ESNUG report. First thing that strikes me is, it seems to suggest in identifying “what assertions to write”. But then it takes a radically different approach to this problem atleast from what has been attempted so far by other EDA vendors. The single most difference is it takes the RTL + Testbench as guide to create assertions/properties. From the report:

 BugScope
takes in our RTL design and testbench as inputs and generates properties,
(which we then categorize as assertions or coverages) that help identify
bugs and coverage holes during simulation. In contrast, Mentor's 0-in
assertion synthesis does not use our testbench;



This is certainly new idea, though I’m little sceptical about the value of late-in-the-cycle assertions.



The next interetsing inference I have on this report is the “coverage property” generation:



When we began our BugScope eval, we only cared about assertion properties
it generated -- we didn't initially see any value of BugScope's coverage
properties.


From what I read in that report, its USP seems to be the “coverage holes” that it can identify. In which case it may be adding more work for the whole project than reducing it – true it helps with better quality, but folks like nuSym will go crazy to have more to cover, but again it is too early to comment in detail. The example given in that report looks little strange as that case maybe due to insufficient run-time of testcase, weak random generation, over-constrained stimulus etc. Also nowadays with RAL (VMM-RAL, www.vmmcentral.org) like automation, all registers can be captured in more controlled fashion from spec. So atleast I fail to see value with the example provided in the report. But since the user says he is using it in production for 2 years or so, there must be credit to this “niche technology”.



Perhaps NextOp is expanding the traditional ABV applications to include “verification closure requirements” by identifying what is not covered yet. That will be interesting application of ABV!



More on this report later.

ABV – points to ponder on its slow adoption

Efforts have been ongoing to make ABV (Assertion Based Verification) more and more deployed for several years via OVL, PSL, SVA etc. Though the concept of assertions is not really new to the industry, widespread usage of it has not been as much as it was expected atleast by the EDA vendors, promoters (to which I consider CVC www.cvcblr,com included).

Prior to PSL/SVA days, 0-in came up with idea of assertion identification, checker library etc. It did catch up with early adaptors but suffered from proprietary solution and inherent limitations of any auto-generated code. This was followed by other EDA vendors developing “auto-generated assertions” for designs – there was some good traction for few quarters and then the initial enthusiasm faded away as the SNR (Signal-to-Noise-Ration) was way too much perhaps.

The development of OVL and other vendor specific assertion libraries looked promising, but IMHO this was not marketed well enough. Also they all fell short of good old 0-in checker elements when it comes to ease of use, verbosity etc. We dealt on this very topic in good detail in our rceent SVA handbook 2nd edition (www.systemverilog.us/sva_info.html) and also touched upon this in our DVCOn 2010 paper (See www.cvcblr.com for downloads page, code, paper + slides are available on request).

  As we at CVC have been walking through these developments in the industry we continue to have debate on what is preventing it from being more widely used. We have several items identified, a non-exhaustive list is below:

  • Who will add these tiny little monsters to start with? Is it RTL designers or Verification engineers?
    • The answer seems to be both.
  • There is a myth that RTL folks don’t want to learn new language – be it SVA/PSL etc.
    • I call it a myth b’cos atleast in this part of the world, the young engineers are always open to new languages, technologies to keep them ahead in technology and beat recession!
    • True, the full PSL/SVA is more than what average RTL guy can consume – but then the kind of properties that RTL folks would write are also simple and don’t require full language capabilities.
    • We at CVC have carefully extracted what RTL designers would require to become productive with ABV – we offer it as 1-day (or even half-a-day if really needed) workshop on “ABV for RTL designers”, see: www.cvcblr.com/trainings or contact us via training@cvcblr.com for details
  • The checker libraries are very handy for RTL folks, but as I said earlier many are not even aware of its potentials. Need more marketing..
    • Some complain about the verbosity especially those who have used 0-in or OVA (inlined) in the past (See AMD’s presentation to Accellera OVL-TC www.accellera.org few years back)
      • Recently released SVA-2009 LRM does address this well with inherited clocks, default clock etc. See www.systemverilog.us for more
      • Also look at checker..endchecker construct in SVA-2009
    • Many users may indeed benefit from a simple “drag-n-drop” style such as the one being developed by ZazzOVL (www.zocalo-tech.com) We at CVC have done initial eval and results look very promising. True, they have some way to go before satisfying every possible user, but good first step I must say!
  • In My design, what assertions can I add?
    • This seems to be much more prevelant question than the myth I mention earlier. There is good element of truth in this concern – only with experience does one get to “identify” quality assertions.
    • There are tools emerging in this space such as NextOp’s Assertion synthesis @:http://www.deepchip.com/items/0484-01.html and Zocalo’s “Zazz bird dog” (www.zocalo-tech.com)
  • How do I know whether my assertions themselves are correct?
    • See: http://www.cvcblr.com/blog/?p=132 for a lively discussion on this topic with Jasper’s ActiveDesign seemingly addressing this well along with other EDA vendors too.
    • Also tools like VCS, Verdi etc. allow assertion evaluation based on a given DUMP file – say VPD, FSDB etc. This is yet another useful feature that’s least marketed – if any. Do look in the tool doc or contact your vendor for more on this, or send us an email via: info@cvcblr.com for more on this.
  • How do I know my assertions really fired?
  • How many assertions are enough for my design?
    • Excellent/Best question perhaps, so NO ANSWER :-)
    • More pragmatically though, there is some research going on at IIT-Kharagpur on this topic, see: http://www.smdp.iitkgp.ernet.in/publications.htm
    • 0-in addressed this with MSD – Minimum Sequential Distance, look in their doc for more
    • VCS adds a stats on “assertion density” – some indications atleast
    • If you are a Masters graduate or PhD – excellent topic to work on!

Twitter of RTL design – welcome to Behavioral Indexing!

Srinivasan Venkataramanan, CVC Pvt. Ltd. www.cvcblr.com

Ajeetha Kumari, CVC Pvt. Ltd. www.cvcblr.com

If you haven’t heard of Twitter you perhaps are living in an internet vacuum J On a positive note, the reach and impact of SNS (Social Networking Sites) into our internet life is hard to ignore – whether it is Twitter, Facebook, LinkedIn etc. To me, a successful SNS tries to capture “what is in going on in your mind right now”? A similar approach can be applied to RTL design – when a designer makes an assumption about the latency of output or the FIFO size etc., it hardly gets captured in a repeatable, executable format. True, at the end of a design phase documentation is written (usually) that attempts to capture these. However it gets too late by then to be “active comments”.

From a language perspective SystemVerilog allows assertions & functional coverage (covergroup) inline with RTL code that can help to some extent. However they are only the “specification” part. A lot more “information” gets lost during such translation such as

· “show me a proof/witness/waveform” for such an occurrence

· Can we optimize the latency to say 5

· What-if I change the FIFO size to 32 here etc.

Jasper’s recently announced ActiveDesign technology has a significant component for this “design process”. It is called “Behavioral Indexing”, you “index” the behavior with facts, assumptions, traces, bugs etc. all in a comprehensive database along with your RTL. So when a designer (or another designer who inherits, reviews the code) looks at the code again (via the ActiveDesign database of-course) he/she can get not only the assumptions (that would be similar to SVA) but also real traces, potential issues of changes to FIFO size etc. In a generic sense the indexing captures the designers state of mind “at that point in time” as a snapshot and keeps it reproducible throughout the lifetime of the RTL code! A good thinking indeed, this is why I like to call it the “Twitter of RTL design”.

There is more to Behavioral Indexing than this, will talk about it next time around, so stay tuned!

Wednesday, March 10, 2010

Introducing “totally vacuous” assertion attempts

See our interesting Blog post at: http://www.vmmcentral.org/vmartialarts/?p=1130 

On the topic of adding SystemVerilog “bind files” – a new tool that is shaping up can help automate even that part – see ZazzOVL (www.zocalo-tech.com). Though as of now it works only for OVL, technically speaking it is very easy to extend it for user specified assertion libraries/modules/MIPs etc.

Sunday, March 7, 2010

Identifying transactions faster with Verdi

Further to our previous blog entry on Verdi’s advanced Transaction debug features (ref: http://www.cvcblr.com/blog/?p=130 ), here are some more tricks that can help debug automation even further.

 

Very often designers find that there are certain unique characteristics/attributes that differentiate transactions. For instance Transaction kind being ERROR/SPLIT/RETRY etc. Wouldn’t it be nice if on a 50,000 clock cycle simulation dump one can:

  • Quickly identify the “information” from “raw data” of signal toggles? (see: http://www.cvcblr.com/blog/?p=130)
  • On top of previous item, classify the transactions based on user specified “attributes/fields” such as ERROR/SPLIT/RETRY

This is very handy trick with Verdi. Select Message –> Filter/Colorify as in screenshot below:

 

verdi_how_2_color

 

Once you are there, define the attributes in the dialog/pane as shown below:

verdi_how_2_color_2

 

Voila! You get:

verdi_colour

 

Add with the signals involved:

verdi_colour_show

 

Now, that’s true “debug automation” and “raising debug abstraction level” in pragmatic sense!

Tuesday, March 2, 2010

Smart application of vmm_log::disable_types()

Srinivasan Venkataramanan, CVC Pvt. Ltd.

Ever wonder why typical SystemVerilog base classes are bulky and seem to make life complicated against simple things like $display? The devil lies in detail – true simple $display is the easiest to use, but think about the code you are writing to have longer life and reuse – then you slowly start realizing the need. Sometimes you need to get bitten by the downside of not using amethodology to start appreciating the need for methodology.

See: http://www.vmmcentral.org/vmartialarts/?p=1098 for a smart usage of vmm_log::disable_types() method. Many folks have asked me why the VMM_LOG is so bloated (in their view), the above is just a sample, see more @ http://www.vmmcentral.org/vmartialarts/?p=259

Monday, March 1, 2010

Look ma “No RTL, TB, only PSL/SVA – yet I can validate my spec”

It is one of those most commonly asked questions in any assertions training/engagement – assertions describe design behavior, but how can I validate my assertions even before my RTL and/or TB is ready? This is useful for few reasons:

1. Users new to writing assertions using PSL/SVA are expected to make mistakes in assertions initially. So it will be good to get some quick waveforms and validate the assertions in stand alone fashion.

2. While the spec is vague, it creates a mechanism to develop/solidify the spec itself

Typically during the advanced sequences/SEREs section of our PSL (http://www.cvcblr.com/trng_profiles/CVC_LG_PSL_profile.pdf) trainings or SystemVerilog Assertions trainings (http://www.cvcblr.com/trng_profiles/CVC_LG_SVA_profile.pdf) this question invariably comes up.

In principle this is a sweet spot for formal tools (Model checkers) – though not all of them openly promote this nice “hidden gem”. I had first hand experience of doing it via some scripts inside Synopsys not too long ago. But not every user is made aware of this “non-productized” feature.

 

Here is what Jasper had to say on this:

The idea you have mentioned about visualizing assertions in SVA/PSL without DUT/TB has been around within Jasper, for a long while. At that time, one of our customers is trying to train more people within their company to write PSL, and many of the engineers found it easy to write "incorrect PSL", and would like us to provide a tool to help them understand whether the PSL they have written captures the intended behaviors.

I also heard atleast one another EDA vendor confirming its tool being able to easily do this. I will add that name if I get clearance :-)

So as my co-author Ben Cohen (www.systemverilog.us) always insists – YES one can (and perhaps should, for a class of designs/specs) use assertions in SystemVerilog (SVA) and/or PSL during the micro-architecture specification stage itself.

Welcome to the new world of possibilities – true this capability has been around for few years, but I haven’t seen many fully exploiting it yet.