In a previous post, Unit testing with Visual C++ 2012, the unit testing features of Visual Studio 2012 were shown to be an effective way to improve software quality by introducing various tests. Still, instead of having to run these tests inside the Visual Studio IDE, it would be even better to integrate these tests into a release management software, so that they are automatically performed during each build of the system.
For running these unit tests outside the Visual Studio IDE, from this MSDN page we find out that there are three ways to run them from the command line:
VSTest.Console.exe: we can use the VSTest.Console.exe program to run automated unit and coded UI tests from a command line. VSTest.Console.exe is optimized for performance and is used in place of MSTest.exe in Visual Studio 2012.
MSTest.exe: we can use the MSTest.exe program to run automated tests in a test assembly from a command line. MSTest is used for load tests and for compatibility with Visual Studio 2010 test projects. MSTest can also be used to view the test results from these test runs, save the results to disk, and save your results to Team Foundation Server.
TCM.exe: Tcm.exe is a command-line utility that lets you perform the following tasks:
- Import automated tests into a test plan
- Run tests that are part of a test plan from the command line
- View a list of test items and their corresponding IDs to use when you import tests or run tests
You can also use tcm.exe to run test cases with associated automation from the command line using a test environment.
Now that we know how these unit tests can be run without opening Visual Studio, we turn to the release management software. In this example we will use SmartBear’s Automated Build Studio, which “easily automates your entire build, deployment, quality assurance and release processes“. ABS includes support to all compilers that are included in Microsoft Visual Studio 2012, 2010, Microsoft Visual Studio 2008, Microsoft Visual Studio 2005 and Microsoft Visual Studio .NET 7.x:
- Visual C# .NET
- Visual Basic .NET
- Visual C++ (both managed and unmanaged projects and modules)
- Visual J# .NET
- Visual F# .NET
Let’s start with a Visual C# project: the AskWatch solution contains both the project of main software app (AskWatch) and the project that contains the unit tests (AskWatchUnitTest)
Running the unit tests inside the Visual Studio IDE:
Now that we have a complete solution, hosting both the software code and the unit tests, we can open Automated Build Studio and create a new build process. In this short demo, we will not include some necessary steps for a complete build process, such as checking out code from a SCCM system, e.g. SVN or TFS, or building the installer, as we are only interested on running unit tests during the automated build.
The build process includes 3 steps:
- building the unit testing project
- running unit tests using MSTest
- building the release of the software app
Each steps depends on the success of the previous step, so if the build of the unit test project fails, the unit tests will not run, and if the unit tests fail, the final release of the software app will not be built. Even if Microsoft suggest using VSTest.Console instead of MSTest, ABS includes a wizard for MSTest but not for VSTest.Console, so we will keep it easy and use MSTest.
The first step is building the unit test project:
- in the Project edit box, select the project file containing unit tests,
- select Debug as Configuration
- select the solution holding both projects in Solution (or ABS will create a dummy one)
- check both Load settings before compiling and Compile via MSBuild
The second step is running the unit tests: select the DLL assembly that contains the units tests in the Debug output directory
Finally, if unit tests are successful, we build the main software app using Release settings:
Now it’s time to run this macro:
All three steps are completed successfully, and the Log window displays the results of unit tests and warning messages of the C# compiler. This macro is now ready to be integrated into the main release management and run each time an automated build is triggered.