[Please welcome Markus Winterholer to the Team ESL blog. Markus is one of the founding members of the ISX R&D team and is from Tubingen, Germany.]
At the Embedded World Conference in Nuremberg, Germany I delivered a presentation with the title "Metric Driven Functional Verification of Embedded Software" which caused a lively discussion started by Assistant Professor Dr. Winfried Dulz from the University of Erlangen asking if we are doing verification or validation. Since I have been doing hardware verification for 10 years I was pretty sure that I knew what verification meant. For three years I have been working on ISX, a product that extends our verification capabilities for embedded software developers. Until now we have focused on using software for the purpose of hardware verification, but our tools and methodologies are also suitable for pure software testing. What is still missing is a common language to address software engineers who never heard of Cadence before (only ten percent of the audience knew anything about Cadence).
We are looking at verification from another perspective, we are not yet speaking the same language, but to be successful in verifying todays complex distributed systems we have to bring in our hardware verification experience and our view of the world and get the software developers with their experience on the same page to discuss and share experiences. Both worlds are working on complex highly parallel systems where timing in communication is crucial. Both are starting the verification and test process by defining requirements and are extracting test cases, checks, and coverage from the specification. Software engineers often use the V-Model to describe the development and test process.
Jason Andrews (Andrews, J.R. 2005. Co-verification of Hardware and Software for ARM SoC Design. Elsevier.) defines the terms Verification and Validation as follows:
Embedded system verification refers to the tools and techniques used to verify a system does not have hardware or software bugs. Software verification aims to execute the software and observe its behavior, while hardware verification involves making sure the hardware performs correctly in response to the outside stimulus and the executing software. (Does it work?).
Validation of embedded systems refers to the tools and techniques used to validate that the system meets or exceeds the requirements. Validation aims to confirm the requirements in areas such as functionality, performance, and power are satisfied. It answers the question, “Did we build the right thing?” Validation confirms that the architecture is correct and the system is performing optimally.
Given this definition, ISX is capable of doing both. Driving constrained random testcases stimulating the software API and monitoring the software behavior (variables and software states) as well as connecting to the hardware testbench to check hardware states at the same time, makes sure that the functionality of system or subsystem was verified. By adding timing checks and power simulations we can validate that the architecture was implemented correctly. For sure we have to do both verification and validation. The flexibility and scalability of ISX should enable both. ISX can be used on various platforms from simulated RTL processor models, over Instruction Set Simulators to Virtual Prototypes, emulators and prototype boards. Depending on the abstraction level of the used software API it can be used for unit testing as well as for system integration tests.
In conclusion, the correct title of my talk probably should have been "Metric Driven Verification and Validation of Embedded Software".