Hardware verification has evolved into keeping track of a pile of different types of coverage. There is line coverage, expression coverage, toggle coverage, assertion coverage, finite state machine coverage, and functional coverage. There are probably more types I'm forgetting related to low power or analog or something else.
Software verification primarily utilizes code coverage. Tools like GNU gcov and others can instrument code to provide statement coverage, call coverage, and branch coverage. Many companies (including Cadence) have standards for code coverage that must be met before software is shipped. The main advantage of code coverage is simplicity. Code coverage results can be generated with very little effort.
One area that is still relatively unexplored is functional coverage for software. Functional coverage can provide interesting feedback on the behavior of the software such as values of global or local variables, function return values, and function arguments. Functional coverage also provides information about the state of a software program at a certain time and can be used in temporal expressions to collect coverage over time. One of the challenges for functional coverage it that it is design specific. This means that creating meaningful functional coverage takes effort to decide what should be covered and what the results mean. Fortunately, most of this effort can be automated, but it will probably never be 100% automated since it's impossible for a tool to know completely what are the interesting things to be covered. An example of a tool for functional verification of software is FoCuS from IBM. For some perspective on a verification view refer to a presentation by Willie Anderson and Rowland Reed from the DV Club. It has some specific slides on functional coverage related to embedded software.
Just before the holiday break there was a post on the new Specman blog related to e ports. This reminded me of a common question that I get frequently about functional coverage on software. When I present ISX to anybody familiar with hardware verification and the world of EDA they immediately start asking questions about the viability of using e as a verification language for embedded software. All kinds stuff must start swirling around in people's heads about language wars and methodology battles between eRM, OVM, and VMM. When I present ISX to software engineers who are not familiar with EDA I always get very positive responses about what a great idea it is to have a special verification language like e. Anyway, back to the post about e ports. The beauty of specman is that it is not specific to the thing being verified. By the use of ports specman can connect to most anything. For ISX, we connect ports to embedded software functions and variables. Often times this software is running in a logic simulator on a Verilog model of a CPU in an SoC simulation, or on an ISS or SystemC model, or on a virtual machine with absolutely no connection to any other EDA tools. Ports make all this possible and it means Specman can be used as a verification tool for embedded software and verifying complex hw/sw issues that is completely independent from logic simulation. This is a great freedom to have in cases where a complete logic simulator is too heavy for the software verification problem.
ISX uses different kinds of ports to connect to embedded software. Simple ports are used to connect to variables and method ports are used to connect to functions. The beauty is the syntax looks the same as you see in the post I referred to on the Specman blog.
Here's an example of a simple port that is connected to a C variable in a C structure.
p1: inout simple_port of byte is instance;
keep bind(p1, external);
keep p1.hdl_path() == "some_object->p1";
keep p1.external_type() == "char";
In addition to this port syntax that is familar to current e users, ISX provides macros to make life easier. Powerful macros are another great feature of Specman. Look up the gsa_port macro in the ISX section of the Specman user guide to see the details.
A new feature in 8.2 is the monitor port. A monitor port is like an in method port that makes monitoring embedded software very easy. Using monitor ports engineers can learn a lot about what is actually happening in the software and passively collect functional coverage, but this sounds like a good topic for another time.
Ideas about functional coverage for embedded software are always welcome.