Friday, June 29, 2007

Notes On Unit Testing

Lately, I seem to be spending a lot of time writing and updating unit tests. Along the way, I've made a number of observations. The syntax I'm using is taken from NUnit, but the ideas apply to any unit test framework.

1) If you choose to Ignore a unit test, note why it's being ignored and optionally how it is to be addressed. In NUnit, the attribute looks like this:

 [Ignore("We can't run this test until the hardware is in place")]

Not only is it easier for a dev to figure out later, but it will show up in the build report, making it easy to see the reason without opening up the code.

2) Similarly, use comments in all Assert statements noting the reason for the failure. Again the NUnit syntax

 Assert.AreEqual(expectedAge, actualAge, "Actual age doesn't match expected");

Without this, the NUnit report tells you that 12 didn't match 45, but it doesn't tell you whether you were comparing age or IQ.

3) Never copy/paste code. We all follow this rule in production code, but it is often ignored when writing unit tests. If every test requires a login before doing anything useful, move the login code to a separate method and mark it with a SetUp attribute. If some tests need to fail login, leave off the Setup attribute, but call the method from those unit tests that need it. Note also that you can use Asserts in these methods, so you can still verify proper execution within the methods.

4) Use constants whenever appropriate. If every unit test accesses the same server, move the server name to a string constant. This way, when you later shift to a new test box, you only replace one string.

5) Use a TearDown method to clean up after a test. Largely, this is to reduce the copy/paste of code (see item #3.) This also has to do with not using "try/finally" blocks. It is true that a failed Assert statement is simply throwing an exception. And technically, you could do code cleanup inside a finally block. But this isn't a clean solution (which is why the TearDown attribute exists.) One note: you may have to do some checking within the method. You wouldn't want to dispose a null object.

6) Never hard-code expected values from a database. I've seen numerous unit tests similar to:

 Assert.AreEqual(14, user.ID);

Primary keys are particularly problematic, but any field could cause issues, especially if users (or other unit tests) update values in the table.

One approach is to pull data using some other means. Perhaps by using inline SQL. Perhaps there's another stored proc or service that returns the same info. Either method means the tests don't need to be updated simply because the data changes.

A second approach is to use mock objects and avoid real data altogether. Assuming you've coded to an interface, you can use a tool like NMock or Rhino.Mocks to simulate data retrieval.

7) Test for exception cases using the ExpectedException attribute. There's no need to write a try/catch block, just to verify that a particular exception was thrown. ExpectedException can handle that for you, saving time writing code. One argument against it is that you can only test one exception case per test. You could instead have multiple try/catch tests within a single unit test. The problem here is that NUnit logs one error per test. If you have multiple errors, only one will be logged. You won't notice the other issues until the first has been addressed.

And Most Importantly:

8) Fix broken unit tests (Keep the unit tests current.) I've seen instances where unit tests were written to verify current behavior, but were then ignored as the code was modified. A large portion of a unit test's value comes from its ability to monitor code, flagging errors as soon as they are introduced.

Thursday, June 21, 2007

Stopping a COM+ Application Via the Command Line

I'm currently working on a project requiring multiple steps to deploy correctly. Being the efficient (aka lazy) developer that I am, I decided to automate some of the process. One step calls for stopping a COM+ application. I'm assuming there's a command line utility to handle this, but I've not yet found it. I tried consulting a friendly guru, but that didn't help. So, I was forced to write the VBScript below.

To use, create a new file, StopComApp.vbs, and add the following:

 dim objCatalog
set objCatalog = CreateObject("COMAdmin.COMAdminCatalog")

set args = WScript.Arguments
serviceName = args.Item(0)

objCatalog.ShutDownApplication serviceName

To use, run the following from the command prompt (or batch file):

 wscript StopComApp.vbs <AppName>

Sunday, June 10, 2007

A Useful Way To Launch WinDiff

A co-worker recently expressed his annoyance that certain tools installed with Visual Studio didn't have Start menu shortcuts. One in particular was WinDiff. In my experience, however, starting WinDiff by itself was never very useful. Once it was running you still had to browse to the files (or folders) you wanted to compare. I prefer selecting two items, right-clicking, and choosing WinDiff from the menu.

To set this up, create a new file, windiff.bat, with the following:
 "C:\Program Files\Microsoft Visual Studio 8\Common7\Tools\Bin\WinDiff.exe" %1 %2

Save the file to C:\Documents and Settings\<username>\SendTo

To use, select two files in the same folder. Right-click and choose Send To > windiff.bat. This can also be used to diff the contents of two folders.

Monday, June 4, 2007

Pseudo-changesets In SourceSafe

Let's say you make a code change that spans multiple files across several projects. When the change is complete, you need to check in all of the files at the same time. Otherwise, you end up with a broken build. Most source-control systems include the concept of a changeset. As you check out files, they are grouped in a specified set. After the modifications have been made and tested, you check in the changeset as opposed to the individual files. The bad news: SourceSafe doesn't include changesets. The good news: you can work around the limitation.

I previously added the MSBuild.Community.Tasks code to SourceSafe. Let's say I modified code within the XmlQuery task. This also required changing the unit tests. To see all of the relevant checkouts, select the folder common to both projects ('Source' in this case.)

From the menu, choose View > Search > Status Search. In the "Search for Status" dialog, select the options to display files checked out to you, in current project and all subprojects. Click OK.

The search results will look similar to this

Select the necessary files, right-click and choose Check In from the menu. Assuming you selected the correct files, the code should be updated without breaking the build.