Continuous Integration for Visual Studio Load Test via Cruise Control .NET

In the previous post we created a load test for the book store service. We developed a functional test per user scenario for generating synthetic load and walked through the entire process of creating and running the Load Test.

In this post we will see how the Load Test can be executed continuously via 'Cruise Control .NET' (CCNet) and provide immediate feedback via CCNet's portal.

here's a quick video that shows the load tests run from CCNet and the results presented in CCNet's portal.

Throughout the post, we'll register the Load Test to run via CCNet and extend the portal to show a summary of the load test results using the trx file generated by the test.

In the next post, we'll take a step forward and add a custom task to CCNet that will query the database and generate a custom xml summary that includes performance highlights, comparison with previous runs and exception details for all the tests that have failed. We will also extend the portal to present the custom content.

The source code is available here http://bookstoreservice.codeplex.com/SourceControl/list/changesets

Getting Started!

We'll start by installing CCNet from here

Once the installation is complete, we'll add the appropriate project/tasks to CCNet configuration file located here: %Program Files%CruiseControl.NET\server\ccnet.config

<cruisecontrol xmlns:cb="urn:ccnet.config.builder">

   <project name="LoadTesting-BookStoreService">

    <!-- Run tests every 4 hours-->

    <triggers>

        <intervalTrigger name="continuous" seconds="14400" 

        buildCondition="ForceBuild" initialSeconds="30" />

    </triggers>    

    <workingDirectory>C:\CodePlex\BookStoreService</workingDirectory>

    <tasks>

      <exec>  

        <executable>DeleteResults.bat</executable>

        <description>Delete previous results</description>

        <!-- Timeout after 1 minute-->

        <buildTimeoutSeconds>60</buildTimeoutSeconds>

      </exec>     

      <msbuild>

        <executable>

            C:\WINDOWS\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe

        </executable>

        <projectFile>BookStore.sln</projectFile>

        <targets>Build</targets>

        <!-- Timeout after 15 minutes -->

        <timeout>900</timeout>

      </msbuild>

        <!-- Tests -->

      <exec>  

        <executable>

            C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe

        </executable>

        <buildArgs>

            /testcontainer:Tests\BookStoreLoadTest.loadtest /resultsfile:Results.trx

        </buildArgs>

        <description>Run load tests</description>

        <!-- Timeout after 20 minutes-->

        <buildTimeoutSeconds>1200</buildTimeoutSeconds>

      </exec>     

    </tasks>

    <publishers>

      <merge>  

        <files>    

            <!-- Add the result file to the build log -->

          <file>Results.trx</file>  

        </files>

      </merge> 

      <xmllogger />      

    </publishers>      

  </project>

</cruisecontrol>

With the configuration above, CCNet will do the following (every 4 hours or one demand):

1. Delete the results from last run (MSTest doesn't override the results)
2. Build the BookStore.sln
3. Run the load test.
4. Add the result trx to the build log (so we can present it in the portal)

Now. we need to extend the CCNet portal to present the results. We need to:

1. Create xsl that transform the results trx to HTML
2. Copy the xsl to '%Program Files%\CruiseControl.NET\webdashboard\xsl'
3. Add link to the xsl to 'Program Files (x86)\CruiseControl.NET\webdashboard\dashboard.config'
4. Restart IIS

Here's the xsl:

<?xml version="1.0" encoding="utf-8" ?>

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">

  <xsl:output method="html"/>

  <xsl:template match="/">

    <xsl:apply-templates select="/cruisecontrol/build/*[local-name()='TestRun']/*[local-name()='Results']/*[local-name()='LoadTestResult']" />

  </xsl:template>

  <xsl:template match="/cruisecontrol/build/*[local-name()='TestRun']/*[local-name()='Results']/*[local-name()='LoadTestResult']">

    <h2>

      Summary for '<xsl:value-of select="@executionId"/>'

    </h2>

    <xsl:apply-templates select="*[local-name()='Summary']/*[local-name()='TestSummaries']">

    </xsl:apply-templates>

    <h2>

      Transactions

    </h2>

    <xsl:apply-templates select="*[local-name()='Summary']/*[local-name()='TransactionSummaries']">

    </xsl:apply-templates>

  </xsl:template>

  <xsl:template match="*[local-name()='Summary']/*[local-name()='TestSummaries']">

    <table border="1" cellSpacing="0" cellPadding="5" >

      <thead style="text-align: center;">

        <td>Name</td>

        <td>Total</td>

        <td style="background-color: fireBrick; color: white;">Failed</td>

        <td style="background-color: darkblue; color: white;">Duration</td>

      </thead>

      <xsl:apply-templates select="./*" />

    </table>

  </xsl:template>

  <xsl:template match="*[local-name()='TestSummary']">

    <tr>

      <td>

        <xsl:value-of select="@testName"/>

      </td>

      <td>

        <xsl:value-of select="@totalTests"/>

      </td>

      <td>

        <xsl:value-of select="@testsFailed"/>

      </td>

      <td>

        <xsl:value-of select="@averageDuration"/>

      </td>

    </tr>

  </xsl:template>

  <xsl:template match="*[local-name()='Summary']/*[local-name()='TransactionSummaries']">

    <table border="1" cellSpacing="0" cellPadding="5" >

      <thead style="text-align: center;">

        <td>Name</td>

        <td>Count</td>

        <td style="background-color: darkblue; color: white;">Elapsed Time</td>

        <td style="background-color: darkblue; color: white;">Response Time</td>

      </thead>

      <xsl:apply-templates select="./*" />

    </table>

  </xsl:template>

  <xsl:template match="*[local-name()='TransactionSummary']">

    <tr>

      <td>

        <xsl:value-of select="@transactionName"/>

      </td>

      <td>

        <xsl:value-of select="@transactionCount"/>

      </td>

      <td>

        <xsl:value-of select="@elapsedTime"/>

      </td>

      <td>

        <xsl:value-of select="@responseTime"/>

      </td>

    </tr>

  </xsl:template>

</xsl:stylesheet>

Here's the modified dashboard.config:

<?xml version="1.0" encoding="utf-8"?>

<dashboard>

  <remoteServices>

    <servers>

      <!-- Update this list to include all the servers you want to connect to. NB - each server name must be unique -->

      <server 

        name="local" 

        url="tcp://localhost:21234/CruiseManager.rem" 

        allowForceBuild="true" 

        allowStartStopBuild="true" 

        backwardsCompatible="false" />

    </servers>

  </remoteServices>

  <plugins>

    <farmPlugins>

      <farmReportFarmPlugin categories="false" />

      <cctrayDownloadPlugin />

      <administrationPlugin password="Pa$$word1" />

    </farmPlugins>

    <serverPlugins>

      <serverReportServerPlugin />

    </serverPlugins>

    <projectPlugins>

      <projectStatisticsPlugin xslFileName="xsl\StatisticsGraphs.xsl" />

      <projectReportProjectPlugin />

      <viewProjectStatusPlugin />

      <latestBuildReportProjectPlugin />

      <viewAllBuildsProjectPlugin />

    </projectPlugins>

    <buildPlugins>

      <buildReportBuildPlugin>

        <xslFileNames>

          <xslFile>xsl\header.xsl</xslFile>

          <xslFile>xsl\modifications.xsl</xslFile>

          <xslFile>xsl\MsTestReport2008.xsl</xslFile>

          <!-- Updated! Adding xsl for presenting load test results -->

          <xslFile>xsl\MsTestLoadReport2010.xsl</xslFile>

        </xslFileNames>

      </buildReportBuildPlugin>

      <buildLogBuildPlugin />

    </buildPlugins>

    <securityPlugins>

      <simpleSecurity />

    </securityPlugins>

  </plugins>

</dashboard>

We are done!

Now we can start the 'Cruise Control .NET' service or run the executable process from %Program Files%CruiseControl.NET\server\ccnet.exe.

Once the CCNet service/process is started, we can go ahead and trigger a build.The simplest way is to browse to the portal (installed and deployed to IIS during the installation) and force a build.

Once the build is complete, we can see the results here:

Another way to trigger a build is using the CCtray client application. The application can be installed from the portal.

CCTray installation can also be found here: '%Program Files%\CruiseControl.NET\webdashboard\cctray'

Once the installation is complete, the CCTray application will be available in the Task Bar (Notification Area). We need to add the project, right click and select 'Force Build'.