In my 2004 book, Co-Verification of Hardware and Software for ARM SoC Design, I wrote about the concept of a co-verification engineer. It's the very last section of the book. Although a lot of people told me they read the book (and some actually learned something from it), nobody ever contacted me and said they were a co-verification engineer, so I'll ask again in a slightly different way in 2008.
For many years EDA companies providing verification products (like Cadence) have been trying to expand the market by selling products that target the embedded system software that accompanies the hardware design and verification projects they already serve. Logically, it sounds like a good opportunity for market expansion. Unfortunately, efforts in this area have been difficult for various reasons:
- Software teams have much smaller budgets to spend on tools, they often use open source software. There is some market research showing this is changing and software teams are now spending more on tools per engineer.
- Software is viewed as less critical since it can always be changed at any time with none of the costs associated with changing hardware. This also seems to be starting to change because delays and software support costs are becoming large enough to get management's attention.
There are probably more reasons like management structure and project time line issues, but the primary obstacle to creating products targeting embedded software might be that there is no concept of a software verification engineer. Software engineers certainly try to produce high quality software using techniques such as code inspections by project peers and other types of testing.
The current situation seems very close to the situation in hardware verification in the mid to late 1990's. Sales people and application engineers tell war stories about Specman being introduced to the market as a verification tool at a time when there wasn't a strong concept of a verification engineer. Most hardware design engineers were also responsible for verification.Their tools consisted primarily of a logic simulator and a waveform viewer. The new thing at the time was code coverage (as a 3rd party PLI program). Explaining the concept of a separate tool and language for verification didn't sink in immediately with engineers who would say things like, "I have a simulator and a waveform viewer, what else could I possibly need?". To succeed with Specman, it was necessary to explain the need for a separate person called a verification engineer with a focus on verification and a different skill set from design. Over time this concept took hold and today not many people question the need for verification specialists.
The next phase of verification evolution might be to create equivalent specialists for embedded software verification. Today's software engineers are much like the hardware designer a decade ago. They says things like, "I write tests for my software in C, I have a debugger if things don't work, line coverage to measure quality, and some scripts to run the tests, what else could I possibly need?".
The role of the software verification engineer is much more than testing, it involves the same concepts that are applied to hardware verification. These are verification planning, constrained random generation to create multiple test scenarios, checking results, and collecting and measuring functional coverage. The job of the software verification engineer is to find bugs in the software, but to use metrics to show the likelihood of bugs is low.
I'm starting to see some job openings on places like monster.com for software verification engineers. I think many are in safety critical domains like aerospace and automotive.
Does anybody out there consider themselves or know people who call themselves a "software verification engineer"?
Maybe I'm completely off and there is no need for rigorous verification techniques for software. After all, the software industry is far older than the RTL design and verification industry and if there was a need it probably would have been solved by now. Maybe the pain or necessity of such methodologies is just not there for software. If you think this is the case, don't be shy and post a comment with your opinion.