Coverage reports in Jenkins

In the current project we use Visual Studio Professional 2013. It can’t give me any code coverage. We also use Jenkins as build server and it would be nice to see a code coverage report after each build. Especially since we have a goal to increase code coverage.

I’ve been experimenting quite a lot to get this working. I have read many useful blogpost and Stackoverflow answers, but for some reason I couldn’t get it to work. But then, by some magic, I decided to read the documentation of one of the tools. And there it was, the solution to all my problems… regsvr32!

I decided to use OpenCover to get a coverage report. Then using ReportGenerator to generate nice reports based on the output from OpenCover. You also have to install HTML Publisher plugin for Jenkins to publish the generated report.

To install OpenCover either you follow the instructions on the wiki on the projects github, or go to the release catalouge and download the installer. Install it on your Jenkins server. For ReportGenerator go to its release catalog and get the zip file. Extract it and put it in a folder on the Jenkins server. I assume that you have the other plugins to run unit tests already installed in Jenkins.

Previously I used VSTestRunner plugin for Jenkins to run unit tests. Now, this build step has to be removed and replaced with a Execute Windows batch command. Here one wrapp the call to VSTest.Console.exe with the call to OpenCover.exe like this:

C:\JenkinsTools\OpenCover\OpenCover.Console.exe -target:"C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" -targetargs:"D:\Jenkins\Workspaces\TestProject\bin\Debug\Tests.dll /Platform:x86 /Framework:framework45 /Logger:trx" -output:"CodeCoverage\TestProject_Build-%BUILD_ID%.VSTest.Coverage.xml" -mergebyhash -skipautoprops

I had to get this to work properly. At first I saw errors when OpenCover commited its work. It said that it couldn’t find the PDB files, or that I had to register a user. So I added -targetdir:"D:\Jenkins\Workspaces\TestProject\PDB\" where I stored all the PDB:s for the moment, and the -register:user flag to register/de-register the profiler. It work perfectly in the command prompt. But I couldn’t get it to work when running from Jenkins. So, checking the documentation for OpenCover it said:

you pre-register the profiler DLLs using the regsvr32 utility where applicable for your environment.

regsvr32 x86\OpenCover.Profiler.dll
regsvr32 x64\OpenCover.Profiler.dll

And now it works!

So, on to generating reports based on the output from OpenCover. From the same Windows Batch command as above, add the following to generate a summary and a report.

C:\JenkinsTools\ReportGenerator\ReportGenerator.exe -reports:"CodeCoverage\Build-%BUILD_ID%.VSTest.Coverage.xml" -targetDir:CodeCoverageHTML -reporttypes:Html

C:\JenkinsTools\ReportGenerator\ReportGenerator.exe -reports:"CodeCoverage\Build-%BUILD_ID%.VSTest.Coverage.xml" -targetDir:CodeCoverageHTML -reporttypes:HtmlSummary

Add a post-build action to publish a HTML report like this:

Post-Build Action - Publish HTML Report
Post-Build Action – Publish HTML Report

Now you’ll have a link to the generated report from the status page of your project.

Additional reading:
Continuous Delivery – Adding Static Analysis