Unit testing with Visual C++ 2012

The article related to multi-threading and SSE2 optimizations (you can find it here) uses a quick-and-dirty way to check if the optimized code is correct, i.e. it runs an iteration on a given input image with a reference serial code, stores the resulting output image, runs an iteration of the optimized code on the same input image, checks if the output image matches that obtained with the reference code. This is a valid approach, but the location is clearly wrong: even if we are writing a demo application, the testing code should be separate in unit tests that can be automated and repeated before check-ins and builds. It’s time to modify that code to use the awesome support for unit testing contained in Microsoft Visual C++ 2012.

Below you will find the full source code of unit tests (and here you can download the complete Visual C++ project), but before reading that code, let’s see which steps are required to get there:

1. every image processing routine must be extracted out of the main C++ file and put in a separate C++ source/header file (named TBBDemoRoutines.cpp and TBBDemoRoutines.h), so that the code that we want to test is separate from the other stuff
2. create a new Visual C++ solution using the wizard for a unit testing project for native code
3. add the files containing the code to test (TBBDemoRoutines.[cpp,h]) to this new project, and set the project properties so that it finds the required library (Intel TBB)
4. the Visual C++ wizard creates a skeleton of the unit testing implementation that we need to flesh out: we have a class named TBBUnitTest defined with TEST_CLASS, and a method named TestTBB1 defined with TEST_METHOD. Each TEST_METHOD should test an aspect of a TEST_CLASS, so in this unit testing project we will define a TEST_METHOD for every possible implementation of optimized image loops, using Intel TBB, SSE2 or both.
5. most TEST_METHODs are likely to share an great deal of initialization and finalization code, so it is not useful to include such code in every TEST_METHOD definition, but it is recommended to create a TEST_METHOD_INITIALIZE (where initialization code is run before calling each TEST_METHOD) and a TEST_METHOD_CLEANUP (whre finalization cod is run after calling each TEST_METHOD). In this demo, TEST_METHOD_INITIALIZE creates both the input (RGBAImage) and output image buffers, fills the input image buffer with random data, runs the reference serial code on the input image buffer and stores the result in an output image buffer (SerialGrayImage), and clears the other output image buffer (ParallelTBBGrayImage); TEST_METHOD_CLEANUP deallocates the image buffers so that there are no memory leaks. The sequence of calls is as follows:
   • TEST_METHOD_INITIALIZE(SetupBitmaps)
   • TEST_METHOD(TestTBB1)
   • TEST_METHOD_CLEANUP(FreeBitmaps)
   • TEST_METHOD_INITIALIZE(SetupBitmaps)
   • TEST_METHOD(TestTBB2)
   • TEST_METHOD_CLEANUP(FreeBitmaps)
   • TEST_METHOD_INITIALIZE(SetupBitmaps)
   • TEST_METHOD(TestTBB3)
   • TEST_METHOD_CLEANUP(FreeBitmaps)

   and so on…It is also possible to use TEST_CLASS_INITIALIZE and TEST_CLASS_CLEANUP, so that the sequence of calls becomes:

   • TEST_CLASS_INITIALIZE(SetupBitmaps)
   • TEST_METHOD(TestTBB1)
   • TEST_METHOD(TestTBB2)
   • TEST_METHOD(TestTBB3)
   • …all other testing methods
   • TEST_CLASS_CLEANUP(FreeBitmaps)

   This sequence does the initialization of reference images just once, so it will run faster, but I recommend the other approach, as it clearly separates every single test method from the other ones, and no corruption of shared data from a previous test method can affect the result of the currently evaluated test method.

6. the body of each TEST_METHOD is very short: it calls a specific version of the optimized code, and then compares the obtained output image with the reference one inside an Assert::IsTrue check: if the image buffers match, the test will be passed, if not, the test will fail. In this demo, these is only one check, but usually, when dealing with more complex data structures and/or data groups, there are multiple Assert::X calls, each checking for a specific error.
7. compile the solution, and if compilation was successful, invoke the unit tester by clicking on Test | Run | All tests, after a few seconds the Test explorer window will show which tests have passed and which ones did not. The unit testing project is now complete, and ready to be integrated in your software building process.

// Intel TBB Demo // by Stefano Tommesani (www.tommesani.com) 2013 // this code is release under the Code Project Open License (CPOL) http://www.codeproject.com/info/cpol10.aspx // The main points subject to the terms of the License are: // -   Source Code and Executable Files can be used in commercial applications; // -   Source Code and Executable Files can be redistributed; and // -   Source Code can be modified to create derivative works. // -   No claim of suitability, guarantee, or any warranty whatsoever is provided. The software is provided "as-is". // -   The Article(s) accompanying the Work may not be distributed or republished without the Author's consent  #include "stdafx.h" #include "CppUnitTest.h"  #include <TBBDemoRoutines.h>  using namespace Microsoft::VisualStudio::CppUnitTestFramework;  namespace TBBDemoUnitTest {             const int DEFAULT_IMAGE_WIDTH = 1 * 1024;     const int DEFAULT_IMAGE_HEIGHT = 1 * 1024;     const int DEFAULT_IMAGE_SIZE = (DEFAULT_IMAGE_WIDTH * DEFAULT_IMAGE_HEIGHT);     const int RGBA_PIXEL_SIZE = 4;      TEST_CLASS(TBBUnitTest)     {     public:         unsigned char *RGBAImage;         unsigned char *SerialGrayImage;         unsigned char *ParallelTBBGrayImage;          TEST_METHOD_INITIALIZE(SetupBitmaps)         {             DisableBenchmarkMode();             // create input image             RGBAImage = new unsigned char[DEFAULT_IMAGE_SIZE * RGBA_PIXEL_SIZE];             // fill RGBA image with random data             srand(0x5555);             for (int j = 0; j < (DEFAULT_IMAGE_SIZE * RGBA_PIXEL_SIZE); j++)                 RGBAImage[j] = rand() % 256;              SerialGrayImage = new unsigned char[DEFAULT_IMAGE_SIZE];             ParallelTBBGrayImage = new unsigned char[DEFAULT_IMAGE_SIZE];             // build a reference gray image that will be compared to those built with multi-threaded code             ProcessRGBSerial(RGBAImage, SerialGrayImage, DEFAULT_IMAGE_WIDTH, DEFAULT_IMAGE_HEIGHT, RGBA_PIXEL_SIZE);             memset(ParallelTBBGrayImage, 0, DEFAULT_IMAGE_SIZE);  //< clear output image so that results from a previous run are zeroed         }          TEST_METHOD_CLEANUP(FreeBitmaps)         {             // free images             delete[] SerialGrayImage;             delete[] ParallelTBBGrayImage;             delete[] RGBAImage;         }                  TEST_METHOD(TestTBB1)         {             ProcessRGBTBB1(RGBAImage, ParallelTBBGrayImage, DEFAULT_IMAGE_WIDTH, DEFAULT_IMAGE_HEIGHT, RGBA_PIXEL_SIZE);             Assert::IsTrue(memcmp(SerialGrayImage, ParallelTBBGrayImage, DEFAULT_IMAGE_SIZE) == 0);         }          TEST_METHOD(TestTBB2)         {             ProcessRGBTBB2(RGBAImage, ParallelTBBGrayImage, DEFAULT_IMAGE_WIDTH, DEFAULT_IMAGE_HEIGHT, RGBA_PIXEL_SIZE);             Assert::IsTrue(memcmp(SerialGrayImage, ParallelTBBGrayImage, DEFAULT_IMAGE_SIZE) == 0);         }          TEST_METHOD(TestTBB3)         {             ProcessRGBTBB3(RGBAImage, ParallelTBBGrayImage, DEFAULT_IMAGE_WIDTH, DEFAULT_IMAGE_HEIGHT, RGBA_PIXEL_SIZE);             Assert::IsTrue(memcmp(SerialGrayImage, ParallelTBBGrayImage, DEFAULT_IMAGE_SIZE) == 0);         }          TEST_METHOD(TestSIMD)         {             ProcessRGBSIMD(RGBAImage, ParallelTBBGrayImage, DEFAULT_IMAGE_WIDTH, DEFAULT_IMAGE_HEIGHT, RGBA_PIXEL_SIZE);             Assert::IsTrue(memcmp(SerialGrayImage, ParallelTBBGrayImage, DEFAULT_IMAGE_SIZE) == 0);         }          TEST_METHOD(TestTBBSIMD)         {             ProcessRGBTBBSIMD(RGBAImage, ParallelTBBGrayImage, DEFAULT_IMAGE_WIDTH, DEFAULT_IMAGE_HEIGHT, RGBA_PIXEL_SIZE);             Assert::IsTrue(memcmp(SerialGrayImage, ParallelTBBGrayImage, DEFAULT_IMAGE_SIZE) == 0);         }             }; }