Writing a Bot to Unit-Test Your Code

 

Introduction

Sometimes we may not always have an environment where we can leverage Visual Studio’s feature for running unit tests after compilation. I observed this situation with a client onsite and decided to implement this function for the system I was building regardless. In my case, I could not leverage this feature as a result of Visual Studio’s license agreement.

Unit-Tests: A conversation between a coder and their machine

Yes. This is my quote. I require unit tests to guide my thoughts as I place my thoughts on the computer monitor. It’s that feedback loop that comforts me when I am unsure of the side-effects that might occur. Imagine tuning a guitar without any feedback from the vibrations that resonates. This is how I feel about my code. Moreover, this is how I feel when I’m implementing or maintaining complex systems. I need that frequency of feedback and guidance.

Implementation

The program that I am providing consists of a WPF application that issues notifications via the notification area on the taskbar. This program checks for updates to an assembly based on a time-interval. In the event that an update to the targeted assembly is recognized, the program executes “mstests.exe”, parses its results, and sends a notification to the notification area.

XAML

The XAML is short and sweet.

This app will be hidden and will not show up in the taskbar.

Note the following property-value pairs in the XAML:

  • ShowInTaskbar=”False”
  • Visibility=”Hidden”
  • WindowState=”Minimized”

The following is the XAML:

<Window x:Class="TestRunner.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Title="MainWindow" Height="350" Width="525" ShowInTaskbar="False" Visibility="Hidden" WindowState="Minimized">
</Window>

 

Code-Behind

The following is the code to implement a bot for unit-testing code:

using AARLGS.TestRunner.Properties;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Drawing;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading;
using System.Windows;
using System.Windows.Forms;

namespace AARLGS.TestRunner
{
    /// <summary>
    /// Interaction logic for MainWindow.xaml
    /// </summary>
    public partial class MainWindow : Window
    {
        #region Members
        string _title = null;
        string _testDllBin = null;
        string _testDllPath = null;
        string _targetAssemblyBin = null;
        string _resultsDirectoryPath = null;
        string _solutionPath = null;

        int _normalWaitInterval = 0;
        int _longWaitInterval = 0;

        int _passNotificationMessageDuration = 0;
        int _failNotificationMessageDuration = 0;

        int _showPassNotificationOnPassOccurance = 0;

        NotifyIcon _notificationIcon = new NotifyIcon();
        int _count = -1;
        bool _lastTestFailed = false;
        #endregion

        public MainWindow()
        {
            InitializeComponent();

            AssignConfigurationValues();
            InitializeNotification();

            var fileCompareList = BuildFileCompares();
            Copy(_targetAssemblyBin, _testDllBin);

            var isModified = true;

            while (true)
            {
                ProcessState(isModified);
                isModified = SetState(fileCompareList, isModified);
            };
        }

        #region Helpers
        private void AssignConfigurationValues()
        {
            _testDllBin = Settings.Default.Test_DLL_Bin;
            _testDllPath = Settings.Default.Test_DLL_Path;
            _targetAssemblyBin = Settings.Default.Target_Assembly_Bin;
            _resultsDirectoryPath = Settings.Default.Results_Path;
            _solutionPath = Settings.Default.Solution_Path;
            _title = Path.GetFileNameWithoutExtension(_solutionPath.Trim('"'));

            _normalWaitInterval = Settings.Default.Normal_Wait_Interval;
            _longWaitInterval = Settings.Default.Long_Wait_Interval;
            
            _showPassNotificationOnPassOccurance = Settings.Default.Show_Pass_Notification_On_Pass_Count;
            _passNotificationMessageDuration = Settings.Default.Show_Pass_Notification_Message_Duration;
            _failNotificationMessageDuration = Settings.Default.Show_Fail_Notification_Message_Duration;
        }

        void Copy(string sourceDirectory, string targetDirectory)
        {
            while (true)
            {
                try
                {
                    Directory.CreateDirectory(targetDirectory);

                    foreach (var file in Directory.GetFiles(sourceDirectory))
                        File.Copy(file, Path.Combine(targetDirectory, Path.GetFileName(file)), overwrite: true);

                    foreach (var directory in Directory.GetDirectories(sourceDirectory))
                        Copy(directory, Path.Combine(targetDirectory, Path.GetFileName(directory)));

                    break;
                }

                catch (Exception ex)
                {
                    var sleeptime = 1000;
                    Debug.WriteLine(string.Format("{0} - Copy Failed\n{1}\nTry again in {2} second...", DateTime.Now, ex.GetBaseException().Message, (sleeptime / 1000)));
                    Thread.Sleep(sleeptime);
                }
            }
        }

        private bool SetState(List<FileHistory> fileCompareList, bool isModified)
        {
            var currentAssembly = Directory.EnumerateFiles(_targetAssemblyBin).Where(f => Path.GetExtension(f).ToLower() == ".dll" || Path.GetExtension(f).ToLower() == ".exe").ToList();
            var currentCodeFiles = Directory.EnumerateFiles(_targetAssemblyBin).Where(f => Path.GetExtension(f).ToLower() == ".cs" || Path.GetExtension(f).ToLower() == ".xaml").ToList();
            var currentFiles = currentAssembly.Union(currentCodeFiles).ToList();
            currentFiles.Add(_testDllPath);

            if (currentFiles.Count() != fileCompareList.Count)
            {
                isModified = true;
            }
            else
            {
                isModified = CheckForModification(fileCompareList, isModified, currentFiles);
                Copy(_targetAssemblyBin, _testDllBin);
            }
            return isModified;
        }

        private void ProcessState(bool isModified)
        {
            if (isModified)
            {
                RunTests();
            }
            else
            {
                Thread.Sleep(_normalWaitInterval);
            }
        }

        private List<FileHistory> BuildFileCompares()
        {
            var previousTestDllInfo = new FileInfo(_testDllPath);
            var currentTestDLLInfo = new FileInfo(_testDllPath);

            var fileCompareList = new List<FileHistory>();
            fileCompareList.Add(new FileHistory() { Path = _testDllPath, PreviousFileInfo = previousTestDllInfo, CurrentFileInfo = currentTestDLLInfo });

            var currentAssembly = Directory.EnumerateFiles(_targetAssemblyBin).Where(f => Path.GetExtension(f).ToLower() == ".dll" || Path.GetExtension(f).ToLower() == ".exe");
            var currentCodeFiles = Directory.EnumerateFiles(_targetAssemblyBin).Where(f => Path.GetExtension(f).ToLower() == ".cs" || Path.GetExtension(f).ToLower() == ".xaml").ToList();
            var currentFiles = currentAssembly.Union(currentCodeFiles).ToList();

            foreach (var file in currentFiles)
            {
                fileCompareList.Add(new FileHistory() { Path = file, PreviousFileInfo = new FileInfo(file), CurrentFileInfo = new FileInfo(file) });
            }

            return fileCompareList;
        }

        private void InitializeNotification()
        {
            var path = @"../../Assets/Failing.ico";

            this._notificationIcon.Icon = new Icon(path);
            this._notificationIcon.Visible = true;
        }

        private void RunTests()
        {
            _count++;

            int passingTests = 0;
            int totalTests = 0;
            int failedTests = 0;

            RunTests(ref passingTests, ref totalTests, ref failedTests);

            if ((failedTests > 0 || _lastTestFailed) || (_count == 0 || _count % _showPassNotificationOnPassOccurance == 0))
            {
                OnNotify(passingTests, totalTests, failedTests);

                if (failedTests > 0)
                {
                    _lastTestFailed = true;
                    _count = -1;
                }
                else
                {
                    _lastTestFailed = false;
                }

                Thread.Sleep(_longWaitInterval);
            }
            else
            {
                Thread.Sleep(_normalWaitInterval);
            }
        }

        private static bool CheckForModification(List<FileHistory> fileCompareList, bool isModified, List<string> currentFiles)
        {
            var filesToAdd = new List<string>();

            foreach (var file in currentFiles)
            {
                var fileHistory = fileCompareList.Where(fh => fh.Path == file).SingleOrDefault();
                var fileExists = fileHistory != null;

                if (!fileExists)
                {
                    filesToAdd.Add(file);
                    isModified = true;
                }
                else
                {
                    fileHistory.PreviousFileInfo = fileHistory.CurrentFileInfo;
                    fileHistory.CurrentFileInfo = new FileInfo(file);

                    var isThisFileModified = fileHistory.CurrentFileInfo.LastWriteTime.CompareTo(fileHistory.PreviousFileInfo.LastWriteTime) > 0;

                    if (isThisFileModified)
                    {
                        isModified = true;
                    }
                }
            }

            foreach (var file in filesToAdd)
            {
                var fileHistory = new FileHistory() { Path = file, CurrentFileInfo = new FileInfo(file), PreviousFileInfo = new FileInfo(file) };
                fileCompareList.Add(fileHistory);
            }

            return isModified;
        }

        private void OnNotify(int passingTests, int totalTests, int failedTests)
        {
            if (failedTests > 0)
            {
                var summary = string.Format("( {0} ) Failed\n( {1} ) Passed\n______________\n( {2} ) Total", failedTests, passingTests, totalTests);
                this._notificationIcon.ShowBalloonTip(_failNotificationMessageDuration, string.Format("{0} Tests Failed", failedTests), summary, ToolTipIcon.Info);
            }
            else
            {
                var summary = string.Format("All ( {0} ) Tests Passed", passingTests);
                this._notificationIcon.ShowBalloonTip(_passNotificationMessageDuration, string.Format("{0}\n", _title, passingTests), summary, ToolTipIcon.Info);
            }
        }

        private void RunTests(ref int passingTests, ref int totalTests, ref int failedTests)
        {
            var buildProcess = new Process
            {
                StartInfo = new ProcessStartInfo
                {
                    FileName = @"C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe",
                    Arguments = string.Format("{0} {1}", _solutionPath,  @"/p:Configuration=Debug /p:Platform=""Mixed Platforms"""),
                    UseShellExecute = false,
                    RedirectStandardOutput = true,
                    CreateNoWindow = true
                }
            };

            buildProcess.Start();

            var output = buildProcess.StandardOutput.ReadToEnd();

            var buildSucceeded = output.Contains("Build succeeded");

            if (!buildSucceeded)
            {
                return;
            }

            var testProcess = new Process
            {
                StartInfo = new ProcessStartInfo
                {
                    FileName = @"C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe",
                    Arguments = _testDllPath.Replace(_testDllPath, '"' + _testDllPath + '"'),
                    
                    UseShellExecute = false,
                    RedirectStandardOutput = true,
                    CreateNoWindow = true
                }
            };

            testProcess.Start();

            var stringBuilder = new StringBuilder();

            while (!testProcess.StandardOutput.EndOfStream)
            {
                try
                {
                    var line = testProcess.StandardOutput.ReadLine();
                    stringBuilder.Append(line);

                    if (line.Contains("Passed:"))
                    {
                        var parts = line.Split(':');
                        totalTests = int.Parse(parts[1].Split('.')[0]);
                        passingTests = int.Parse(parts[2].Split('.')[0]);
                        failedTests = int.Parse(parts[3].Split('.')[0]);
                        break;
                    }
                }

                catch (Exception ex)
                {
                    var message = string.Format("Error parsing results...\nTrying again later");
                    this._notificationIcon.ShowBalloonTip(_passNotificationMessageDuration, _title, message, ToolTipIcon.Info);
                    Debug.WriteLine(ex.GetBaseException().Message);
                }
            }

            var text = stringBuilder.ToString();
        }
        #endregion
    }
}

 

Configuration

This implementation requires that only the configuration file be updated for unit test configuration. This means that the code does not have to be edited and recompiled. Instead, just modify the app config file and launch the app again.

The following is the configuration:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <configSections>
        <sectionGroup name="userSettings" type="System.Configuration.UserSettingsGroup, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" >
            <section name=" TestRunner.Properties.Settings" type="System.Configuration.ClientSettingsSection, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" allowExeDefinition="MachineToLocalUser" requirePermission="false" />
        </sectionGroup>
    </configSections>
    <startup> 
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
    </startup>
    <userSettings>
        <TestRunner.Properties.Settings>
            <setting name="Test_DLL_Bin" serializeAs="String">
                <value>C:\Users\myusername\Documents\Visual Studio 2012\Projects\mySolutionFolder\myTestProjectDirectory\bin\Debug</value>
            </setting>
            <setting name="Test_DLL_Path" serializeAs="String">
                <value>C:\Users\myusername\Documents\Visual Studio 2012\Projects\mySolutionFolder\myTestProjectDirectory\bin\Debug\myTest.dll</value>
            </setting>
            <setting name="Results_Path" serializeAs="String">
                <value>C:\Temp</value>
            </setting>
            <setting name="Normal_Wait_Interval" serializeAs="String">
                <value>15000</value>
            </setting>
            <setting name="Long_Wait_Interval" serializeAs="String">
                <value>60000</value>
            </setting>
            <setting name="Target_Assembly_Bin" serializeAs="String">
                <value>C:\Users\myusername\Documents\Visual Studio 2012\Projects\mySolutionFolder\mySolution\bin\Debug</value>
            </setting>
            <setting name="Solution_Path" serializeAs="String">
                <value>"C:\Users\myusername\Documents\Visual Studio 2012\Projects\mySolutionFolder\mySolution.sln"</value>
            </setting>
            <setting name="Pass_Notification_Wait_Duration" serializeAs="String">
                <value>600000</value>
            </setting>
            <setting name="Show_Pass_Notification_On_Pass_Count" serializeAs="String">
                <value>5</value>
            </setting>
            <setting name="Show_Pass_Notification_Message_Duration" serializeAs="String">
                <value>250</value>
            </setting>
            <setting name="Show_Fail_Notification_Message_Duration" serializeAs="String">
                <value>3000</value>
            </setting>
        </TestRunner.Properties.Settings>
    </userSettings>
</configuration>

 

Conclusion

In conclusion, I have described the implementation for writing a bot to unit test code. The program is implemented to compile once. Thus, settings modifications can be set within the app’s config file.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: