Integrating Coco Code Coverage with Unit Test Frameworks

Increasing demands on the quality of software applications has bolstered the need for adequate, thorough testing, including, but not limited to, at the unit level. Integrating code coverage analysis tools with your unit test framework gives a clear sense of the quality of your tests: the grade to which the code has been “hit” by your tests, how often or how little a part of the code has been executed, and what the coverage looks like as your testing efforts evolve.

Squish Coco offers a flexible, open approach to the integration with unit test frameworks. In principle, every unit test framework is supportable.

Custom Unit Test Frameworks

We will demonstrate Coco’s ability to integrate with a generic unit test framework. In this example, we use the C++ programming language. As a first step, let’s understand how the integration works.

As a code coverage tool, Squish Coco is inherently not aware (without additional help) of which parts of the code are tests and which parts are not, because unit tests are viewed as parts of the code from Squish Coco’s perspective. Therefore, some initial work is required.

In order for Coco to distinguish a test from a) other tests and b) other parts of the code, we must provide Coco with the name of the test and clearly state: “this is the beginning of the test”, “this is the end of the test”, and “please save the coverage results of the test.”

We do this by calling some special functions:

 __coveragescanner_testname("sometest");

…To provide Squish Coco with the name of the test.

 __coveragescanner_teststate("PASSED");

Or,

 __coveragescanner_teststate("FAILED");

Or even,

 __coveragescanner_teststate("UNKNOWN");

…To provide Squish Coco with the information on whether the test was successful or, if for any reason, the test result remains unknown.

Next, we effectively say that the test has begun, with:

 __coveragescanner_clear();

And that the test has ended with:

 __coveragescanner_save();

These last two functions are automatically generated and injected into your code if Squish Coco is enabled.

Putting the pieces together, we have the following for a particular test:

SOMETEST_FUNCTION()
{
#ifdef __COVERAGESCANNER__
      __coveragescanner_clear();
      __coveragescanner_testname("sometest");
#endif
      // Here comes the testing part, checking if some integer 'i' is even
      if ( i % 2 == 0 ) {
            __coveragescanner_teststate("PASSED");
      } else {
            __coveragescanner_teststate("FAILED");
      }
      // and we are saving the results...
#ifdef __COVERAGESCANNER__
      __coveragescanner_save();
#endif
}

#ifdef __COVERAGESCANNER__ … #endif is a necessary part which keeps the test functioning without Squish Coco (or with Squish Coco disabled.)

A General Approach

Let’s assume we have some framework with an API similar to Google Test. Let’s further assume that in order to use it, we must include the framework.h header file.

We have our main.cpp file as in the following:

#include "test.h"
#include "framework.h"

int main(int argc, char** argv)
{
#ifdef __COVERAGESCANNER__
       // initialize CoverageScanner library
       __coveragescanner_install(argv[0]);
 #endif
       InitFrameWork(&argc, argv); // init our test framework
       return RUN_ALL_TESTS(); // run all tests listed in the project
} 

Our test.h file:

#ifdef __COVERAGESCANNER__
#define COCOTESTBEGIN(NAME) __coveragescanner_clear(); \
__coveragescanner_testname(NAME); \
__coveragescanner_teststate("PASSED")
#define COCOTESTFAILS() __coveragescanner_teststate("FAILED")
#define COCOTESTEND() __coveragescanner_save()
#else
#define COCOTESTBEGIN(NAME)
#define COCOTESTFAILS()
#define COCOTESTEND()
#endif

And test.cpp, as in:

#include "test.h"
#include "framework.h"

 TEST(TestCaseName, TestName1)
{
       COCOTESTBEGIN("testName1");
       EXPECT_TRUE(true);
       // HasFailure() returns true if the test fails in full or in part   
       if ( HasFailure() ) COCOTESTFAILS();
       COCOTESTEND();
}

 TEST(TestCaseName, TestName2)
{
       COCOTESTBEGIN("testName2");
       EXPECT_EQ(1, 1);
       if ( HasFailure() ) COCOTESTFAILS();
       COCOTESTEND();
}

 TEST(TestCaseName, TestName3)
{
       COCOTESTBEGIN("testName3");
       EXPECT_TRUE(true);
       EXPECT_TRUE(false);
       if ( HasFailure() ) COCOTESTFAILS();
       COCOTESTEND();
}

Thus, we have three macros defined: COCOTESTBEGIN, COCOTESTFAILS and COCOTESTEND. The rest is up to you. If you keep the file test.h exactly how it is and ‘#include’ it in compilation units (.cpp files) accordingly, it will be suitable as well for any other framework.

This also works without Squish Coco (or with Squish Coco disabled).

Please do not forget to use a framework’s function similar to HasFailure() which must return ‘true’, in case the test somehow fails (in full or in part), and ‘false’, otherwise. In this example the third test fails, but only partly. Regardless, this will be recorded in Squish Coco as a failure.

Further Reading

Our documentation outlines configurations for certain popular unit test frameworks, like Google Test and CppUnit.

The post Integrating Coco Code Coverage with Unit Test Frameworks appeared first on froglogic.