What’s beyond UVM? - Excerpts from DVCon BoF panel

Last week at DVCon we had a very interesting Birds-of-a-Feather meeting on Mar 1st evening. Panelists included Tom Anderson from Cadence, Yunshan Zhu from NextOp and Adnan Hamid from Breker Systems.

 

It was moderated by Srinivasan Venkataramanan from CVC. See pre-BoF invite/details at:

http://www.cvcblr.com/blog/?p=272

The theme of this panel was “Verification Closure” – given that 2011 DVCon marked the birth/release of UVM standard for developing reusable, inter-operable VIPs, it was a perfect fit for a group of technocrats to explore what is beyond UVM. We had some very interesting discussions and here are some excerpts. If you feel I missed some discussion topic or have any comment on this, feel free to blog it here!

To start with, Rick Nordin of Breker Systems introduced the panel topic and the moderator.

                      

Srini (www.linkedin.com/in/svenka3) presented single slide setting the stage for the panelists and had the honor of introducing the esteemed panelists of the evening: Tom, Yunshan & Adnan.

Here is the slide from Srini:

Basically – we have the UVM now, how do we leverage and verify a complex SoC with it – what are the pieces of the puzzle involved beyond UVM itself? At broad level here are the key pieces:

• Tests/Stimulus – co-ordinated across several UVM VIPs/interfaces – this is far beyond regular SystemVerilog constraints, UVM sequences. Sure UVM’s virtual sequences can help, but with lot of coding and very little reuse from block to SoC level, see http://www.cvcblr.com/blog/?p=272 for some example.
• Targets/coverage goals – not just “code coverage” or “cross every possible input to every possible output” – rather “qualified”, identified and possibly filtered as per existing simulation runs – all automatic
• A robust platform to do the execution, track it in an executable plan and measure the progress

 

Tom presented first with Cadence’s solution of automated constraint automation, vPlan and integrating adjacent technologies such as Formal, Emulation etc. He was as grand as ever in his vision and covered a whole lot in 1-slide indeed!  Pircked it from Tom’s parallel blog on this topic: http://www.cadence.com/Community/blogs/fv/archive/2011/03/11/a-modest-proposal-using-formal-to-close-coverage-gaps.aspx

Then Yunshan presented how to automatically identify qualified coverage targets based on RTL + existing simulation run through NextOp’s Assertion Synthesis. This has caught lot of users excited about this technology and the emerging customer reports from companies like Altera (at deepchip.com) is only making the case more stronger! Here is a quick snapshot of what NextOp brings to the table in this space (from: http://www.nextopsoftware.com/Te_AssertionBasedVerification.html).

 

 

Yunshan also presented on how this fits nicely to today’s existing flows and extends the scope for formal/emulation on demand basis. This is extremely beneficial as the audience got to see that NextOp doesn’t really change the flow, rather fitst nicely to any existing flow and brings in value instantly.

Then it was Adnan’s turn to present his views and he started with a BANG! showing how “Model based stimulus-check-cover” generation helps in achieving Verification Closure. In his terms the “testcase generation” is incomplete if it talks “only” about the stimulus – instead it should capture the “stimulus, expect/checker and a coverage model” all in one. A grand idea in plan, call Breker to see how to make it true on your designs!

Then Adnan moved on to addressing the SoC level challenges in developing, C based tests with all desired bells-n-whistles such as:

• Reuse of IP level scenarios/graphs
• Automated C test generation
• Heavy memory coherency and
• Debug of such scenarios as-if it is plain SW code (via ubiquitous printf)

 

Perhaps too much to absorb in 10-minutes, but that’s why they had a booth at DVCon exhibit :-)

Then we had an exciting set of audience questions ranging from “explicit goals to hidden targets” (an interesting attendee posted it by taking examples from Hindu mythological characters such as Ravana & Indra – I will try and cover that in a separate blog on that for pleasure, if not anything else..). A quick summary of that question:

I am clear of an explicit goal to be hit by my SoC env (though it is not trivial to

hit it through simple set of “constrained random sequences” running on their

own, need a great deal of co-ordination).

But what about those “hidden/invisible” targets that are deeply buried inside

my huge RTL code base?

 

Ans: This is where "Assertion Synthesis” neatly plugs-in – given a set of simulation results, it goes into selected set of blocks/IPs, analyzes it and reports “quality properties” that the design + simulation exhibits. Now the user (typically the block owner) reviews those properties and marks these as assertions or cover holes. Sure there is work – but a targeted set of properties on potentially unstable/suspicious blocks/IPs with automated pruning of “redundant” stuff is a BIG deal indeed. Talk to www.nextopsoftware.com for more!

Some of the attendees:

There were some excited/over heated arguments about temporal properties vs. white-box ones, especially from Vibarajan (http://www.linkedin.com/pub/vibarajan-viswanathan/3/568/343P). The fact is existing System Verilog constructs do a great deal of  work for block level/IP level properties, but they fail to deliver at SoC level of abstraction involving class based transaction level behaviors. Sure one can do lot of code around behavioral checkers/scoreboards (and as my good friend Nitin puts it – aren’t you trying to use wrong feature for wrong problem? – I respectfully disagree on this).

Another interesting discussion was around reach-ability of various cover goals. The panel seemed to have the best answers given the abundence of talent/experience on the panelists. Tom explained how formal analysis can either reach them – even if they are hard to reach via regular constrained random generation. Tom also pitched in the role of “unreacahbility analysis” as some of the goals might be just NO-GO given the design constraints/logic and it is very hard to identify them manually. When it comes to higher level, system level scenarios there is this new Graph based scenario models that can literally “mind-map” verification intent thereby letting users “begin-with-end” in mind. You need to listen to Adnan to believe it, or better yet evaluate their flow to benefit from it!

Photos from this BoF courtesy Joe Hupcey are at: http://www.flickr.com/photos/24605532@N08/sets/72157626233541664/with/5513590114/

If you have any more comments/questions to the panel, feel free to add them here in this blog (http://www.cvcblr.com/blog/?p=319) as comments, I will try and get the panelists respond.

Extracts from DVCon UVM poster session – it is vibrant ecosystem indeed

Here are some snaps from recent DVCon UVM poster session. More than 12 vendors demonstrated their commerical offerings around UVM and it was an electrifying experience for the attendees/potential customers (some 180+ UVM tutorial attendees). Products/offerings broadly fell in the following categories:

• Simulators – Cadence, Synopsys (DOn’t recall Mentor on poster, but of-course Questa supports UVM), another missing poster EDA tool company was Aldec. Though both Mentor & Aldec had their booths at the exhibit.

Cadence had a poster on how IUS supports UVM and extended debug features targeted for UVM users. Here is Joesph H with his Cadence poster

 

 

Synopsys had a clear, simple poster on how VCS extends its leadership in SystemVerilog performance. Here is Adiel Khan explaining with passion to a customer Synopsys poster:

 

• Register model automation, maintenance: AgniSys dominated the poster with lots of visitors asking questions. Infact their poster attracted attendees even after the tutorial break was over and the session re-started. Alas, Anupam wasn’t there, but Srini from CVC (www.cvcblr.com) did some back-up for Anupam. Part of EDA ecosystem, Huh!

Here is Anupam explaining his poster to a customer potential:

Other similar offerings were demonstrated by Semifore & Doulog.

• Trainings: Clearly attendees were well treated by thriving training ecosystem – with CVC (www.cvcblr.com), Doulos and others showing off their new trainings on UVM.

 

 

• VIPs: CVC (www.cvcblr.com) launched its latest campaign around UVM aptly named “UnleashingUVM” with trainings, products & services around UVM.

Here is our CTO, Srini (http://www.linkedin.com/in/svenka3) with the poster:

 

And we had an interesting visitor all the way from home (Bangalore) – Mr. Amit Sharma :-)

 

 

Here is a more detailed list of CVC’s products & Services around UVM:

 

 

 

Overall it was a great successful poster session and the user interest around it is a standing testimonial to the overwhelming customer expectation on UVM.

Why UVM is important for the Semiconductor community?

At DVCon Accellera released its latest standard for VIP interoperability named UVM – Universal Verification Methodology.

12+ vendors demonstrated their commercial solutions around UVM – hasn’t happened for a long time in the industry around single standard – except perhaps for SystemVerilog itself (back in 2003?)

While the technical details can be talked for very long time, here is a practical, real-life experience of “waiting for too long to have this standard”. This could be your simple means of convincing your technical management why they should be looking at UVM seriously in next project.

On the DVCon evening (Mar 1st) I was having dinner with a good old friend of mine, Mahesh. Here are his experiences/pains of working with various verification projects for the past 6+ years:

Mahesh –

it was a nightmare 5-6 years ago for Verification engineers – you move from one project to another within the SAME company (for instance acquired companies in a large company), your way of work – a la - “methodology” changes, hell! How I write my BFM, how it interacts with rest of the environment, how do I control my number-of-transactions etc. etc.

And when you switch jobs – almost guaranteed to go through the unlearn-learn cycle. While this cycle is good in many-a-context, but not to do the same thing – in this case “verification” – just with different base classes/languages.

 

He was listing: e, OpenVera, SystemVerilog languages and eRM, RVM, VMM, AVM and the likes..include OVM if you move the timeline little closer to 2011 :-)

So having a standard methodology is certainly promising for end users and for management to manage resources across projects. Sure there are more compelling technical reasons too – hopefully all for better, but the good things is, even if there are some shortcomings, it is a platform to add things to and bring it to Accellera to grow it beyond UVM 1.0.

Here is a snapshot of what CVC (www.cvcblr.com) has to offer for you around UVM – UnleashingUVM

 

 

Explore technologies for “Verification Closure” at DVCon BoF meeting, Tuesday Mar 1st 6.30 PM (PST)

Big picture – Verification Closure

Panel members: Cadence, NextOp, Breker & CVC

If you are attending DVCon starting tomorrow, here is a panel that you may not want to skip – yes UVM is hot and ready-to-go. How do we leverage that and get to faster Verification Closure – that’s precisely what we will be discussing in this “Birds-of-a-Feather” panel at DVCon www.dvcon.org on Tuesday Mar 1st, 6.30 PM US/Pacific time at Donner Ballroom, DoubleTree Hotel, San Jose.

Here is a summary of what to expect in this panel discussion:

UVM is great! Enables interoperable VIPs to be created, reused. In a typical SoC – several such UVM VIPs get integrated and 1 (or more) embedded processors (ARM-like) configure/control the flow. Individual sequences/virtual sequences at UVM level will do great for peripheral-alone testing.

Taking right from UVM SoC reference flow @ www.uvmworld.org, here is a sample SOC:

How about true “flow/scenario” testing? UVM’s virtual sequencer is “A possibility”. A pragmatic approach as outlined by an excellent article by my good friends @ Applied Micro, Pune (India) is here:

http://www.design-reuse.com/articles/22264/system-verilog-ovm-verification-reusability.html

 

 

The above approach requires lot of coding, synchronization and pretty much directed across transactions/interfaces. How do we randomize “across interfaces/peripherals” to mimic system-level flows/scenarios?

Even if we code up all Sequence libraries, virtual sequences and virtual sequencers – we got only the stimulus, what about complete “Verification Closure”?

On top, overlay Low Power features, requirements and annotate Power State table information – the number of different paths/arcs to be coded and tested is mind boggling – imagine coding them via virtual sequences/sequencer – do-able, but lot of work indeed!

Every DV team does this today in one-way or the-other. But what new technologies are available or becoming available to assist?

Come and listen to experts in this domain at “Birds-of-a-Feather” panel at DVCon www.dvcon.org on Tuesday Mar 1st, 6.30 PM US/Pacific time at Donner Ballroom, DoubleTree Hotel, San Jose.

Here are some key items that would be discussed: If you have more ideas/questions send them across to sevnka3@gmail.com or post as additional comments here at www.cvcblr.com/blog I will incorporate them as much as I can!

• Need truly inter-operable VIPs to start with – This is where UVM comes-in. We at CVC are at DVCOn UVM poster session.
• Need key “metrics” to define, drive and track the progress (Various sources including formal)
• How to focus on “critical, high quality” coverage targets?
• A coherent, high-level mechanism to capture the scenario models that aid in:
• generate stimulus
• capture scenario specific checks/criteria for success and
• Cover them across interfaces, temporal transaction coverage

Come and share your views, learn what your fellow DV folks do all at DVCon!

Feedback from customer on our SystemVerilog training

 

Recently TeamCVC (http://in.linkedin.com/in/cvcblr) conducted a 4-day SystemVerilog workshop at Kochi, South India. Some musings at:

http://www.cvcblr.com/blog/?p=259 

And today we received a cool note from customer voluntarily:

Vinayaraj T R, Project Engineer @Cochin:

I have attended your training on System Verilog for Verification conducted last week at Kochi.  The session was very much helpful for me and even being a fresher I was able to understand the concepts and gain a lot of knowledge from it. I am very sure that it will help me a lot through out my career.

It would be helpful if you could share the lab tutorials of that training.

Thank you once again.

Regards,

Vinayaraj T R,

Vinayaraj is certainly not alone. It is this customer satisfaction that gives us the “passion” to do more!

Until another customer success story, it is sign-off from Bangalore, TeamCVC

Find deeply buried functional bugs with Graph based solver

 

Are you an expert Verification engineer using upto date languages & methodologies available such as Specman/E, SystemVerilog, VMM, OVM, UVM etc.? Are you looking for even more technologies to find “deeply buried functional bugs” in a language agnostic manner, yet be able to reuse the underlying TB code? Read what Dave Whipp, a veteran HW Verification engineer found working over the last few years:

http://verificationguild.com/modules.php?name=Forums&file=viewtopic&t=4172 

Specifically:

There is a pressing need for testing tools that are language-agnostic – and such tools are indeed emerging. The shadow of the SystemVerilog steamroller is lifting.
One such tool, that I have been using successfully over the past few years, is Breker’s Trek. Trek randomly generates directed tests using a constrained random walk of a graph (constructed by verification engineers) that describes how an environment interacts with the DUT. This is a step back from the purist approach of SAT-based constraint solvers, but it does provide an effective platform for exploring deep sequences of interactions. It would be good to see panel discussions of SAT-solving Vs graph walking as methodologies for finding deep bugs.

See a snapshot of how a graph based solver can explore constraints that maybe “temporal” and across several transaction hierarchies:

 

 

There is lot more to Trek than this, but this ability itself is beyond today’s existing SAT based solvers. Now combine that with the STRONG and UNIQUE block-to-SoC auto test generation – a new paradigm in verification is rolling out..

See www.brekersystems.com for more. And CVC (www.cvcblr.com) is your partner in India to bring this advanced technology close to you, call us if you want to learn more.

Happy testing!

SystemVerilog Assertions’ field-day at Port city of Cochin!

 

TeamCVC (http://in.linkedin.com/in/cvcblr) is at Cochin, a famous port-city in South India (http://en.wikipedia.org/wiki/Kochi) this week on a “Mission SystemVerilog” at a customer site. It is a 4-day program covering:

• SystemVerilog basics for RTL designers (http://www.cvcblr.com/trng_profiles/CVC_LG_SVD_profile.pdf) 
• System Verilog Assertions (http://www.cvcblr.com/trng_profiles/CVC_LG_SVA_profile.pdf)
• SystemVerilog for Verification engineers (VSV: http://www.cvcblr.com/trng_profiles/CVC_LG_VSV_profile.pdf)

The audience is a mix of young, enthusiastic engineers in their early-to-mid career – all very keen to hone their skills on SystemVerilog. Our CTO, Srini (http://in.linkedin.com/in/svenka3) chose to customize the training in a timely manner to get the audience involved and interactive. During Day-2, it was a true “field-day of SystemVerilog Assertions”. Especially when it came to Sequence repetition operators, it was fun all across the room. Needless to say CVC’s SystemVerilog Assertions labs are very well laid out with concrete examples to demonstrate the various sequence operators. However with assertions the fun really lies when you “slightly” change the sequence and/or the trace. Here are some screenshots from this “field-day”.

One of the sequences that we experimented is to demonstrate the difference bet’n non-consecutive [= N] & GOTO [-> N] operator.

One of their smart engineers asked/wanted to change the ##1 to ##0. This is to explain the “endpoint” of a sequence/property as in:

  a |=> b [-> 2] ##1 c;

 

Lucky that we had access to Riviera-Pro running on our laptop, on-the-fly we could tweak the code/trace and demo it live:

 

The fun gets only more when there are multiple threads & multiple attempts. Here is how Riviera-Pro nicely shows it up on Waveform – like a real “THREAD”.