Test Driven Developer

Unit Testing and Test Driven Development - a Practical Guide

John E. Boal — Sat, 15 Nov 2008 21:08:59 GMT

Here is a presentation I wrote on what Unit Testing is all about, and how TDD fits into the ATDD cycle.

There are specific things here on testing the UI code with Selenium and JSUnit, and recommendations on how to do unit testing on your database code.

This presentation is in PDF format, but I can post the PPTX format also if needed.

A Practical Guide to Unit Testing1.pdf (503.29 KB)

November Agile Beer Night is Nov 20

John E. Boal — Sat, 15 Nov 2008 21:06:29 GMT

For all of you who enjoy beer and talking about Agile Software Development... Agile Beer Night is for you! See the Agile Beer Users Group web site for more information about the meeting location. It's at Daman's Pub from 5PM-7PM.

Test-Driven Process Improvement with Refactoring, Value Stream Mapping, and Windows Workflow

John E. Boal — Thu, 30 Oct 2008 20:47:03 GMT

Value Stream Mapping [VSM] is a lean concept that helps us improve our processes by focusing on the whole process and making stepwise refinements on smaller parts of the process, to gain improvement for the whole. It is in a sense, "Refactoring" a process.

In an ideal case, a process can be thought of as a workflow, and perhaps we might be able to represent it using the WF Windows Workflow Foundation code delivered in .NET 3.5. It would be nice to show a code example here of how we can represent a process with workflow steps, but I didn't have time to write up an example as of yet.

With VSM, we are trying to improve a process by mapping it into parts, each of which we can study. We can gather metrics on each of the process steps, and metrics on how long things wait between steps. The waiting between steps is usually far more significant time-wise than any of the steps themselves. WF can be instrumental in providing an automated framework for collecting and reporting these metrics.

Indeed there are also other workflow alternatives of course. BizTalk Server 2006 R2 is a fine candidate for workflow representation as well, when adapters are available for systems that interoperate. I have a specific Microsoft focus in my career, but I have used CapeClear as a Java-based workflow system, and there are others in the ESB space which have features that would allow us to collect metrics on an automated representation of our processes.

Automation is key to refactoring, as in any development mechanism. We can't fix what we don't know is broken. With VSM, we focus on the entire process as a whole, so we need to collect measurements on where it is today, and then we can see total improvement on the whole as we make small changes (refactoring the process).

Many enterprises today use Service-Oriented Architecture [SOA] as a design pattern for the construction of enterprise systems. This architecture lends itself well to the use of WF or BizTalk (substiitute your favorite flavor here) workflow components. SOA helps us to map the entire process to blocks of functionality that we can measure and monitor easily, contributing to our VSM overall picture.

My theory is that like with software development, process improvement can also have a test-driven, scientific approach. VSM is a tool that helps us change the right things to improve our overall process, but it typically has measurements in large units - days, even weeks. With the right automation framework contributing to very rapid feedback with the metrics collected on processes, we can write tests that indicate Red or Green for not only each component being analyzed, but for the entire process as well.

Tests don't have to be in the form of an NUnit DLL or test fixture. We can use simple reports actually to give us the feedback we need to make decisions on how we refactor. Think of it as moving test-driven concepts a couple elevations higher in the plan view. From the 500 foot view to the 5000 foot view if you will. It's the exact same concept, just applied at a somewhat higher level of abstraction.

For each of our services we monitor the time taken in the process and the time between processes, and compare them to our initial baseline. We might even be able to get real-time data on a visible dashboard in the best of cirumstances. When we are better than our initial baseline, we are Green, if not, then we're Red. There is no yellow - same result means we haven't improved so it's still red. This concept is brutally simple, but it can be very effective, especially if the Red/Green report is made visible on a web-based report.

Mapping the workflow using WF, BizTalk, or other tools gives us a way to take automated measurements and compare them to our baseline to render a decision on whether our process has improved or not, and whether we are reaching our overall goal of total improvement. Individual portions of the whole can still be red as long as we achieve green for the overall process.

A business process really is a workflow, we just need to map it as such using software and automation. In this way we can apply the fine concepts of test-driven and refactoring to process improvement, once again using automation and measurement as our top key tools for leveraging our own technologies to help us improve them.

Code examples here would be good of course... Stay tuned for future posts with WF code examples of a process flow that has metrics and baseline comparisons for Red/Green.

More Tips for Unit Testing in Iron Python

John E. Boal — Tue, 07 Oct 2008 16:09:49 GMT

Do you ever wonder why there is an extra "False" printed at the end of each run for unit tests?

C:\Code\sample>ipy UnitTests.py
.....
----------------------------------------------------------------------
Ran 5 tests in 0.395s

OK
False

It looks like there is an exit statement buried inside the unit test framework. We can eliminate the problem pretty easily with the addition of a try/except around the unit test execution. Here is the code to put at the end of your unit test script:

if __name__ == '__main__':
    try:
        unittest.main()
    except SystemExit:
        pass

This catches and eats the SystemExit exception, and runs without printing the "False" at the end of the execution:

C:\Code\sample>ipy UnitTests.py
.....
----------------------------------------------------------------------
Ran 5 tests in 0.387s

OK

cheers!

Unit Testing in Python

John E. Boal — Tue, 30 Sep 2008 13:26:26 GMT

I have been using IronPython lately, and am having a good time testing code with its built-in unit test framework. Here is a brief example of how to write some unit tests for IronPython (or CPython if that's your flavor). These unit tests leverage the "unittest" module available in Python 2.2 and above.

Here is a short test class:

class TestCode:
    localInt = 1
    @classmethod
    def Method1(self):
        return 100
    def Method2(self, a, b):
        self.localInt = a * b

Here are some sample unit tests for it, using the unittest framework:

import unittest
class UnitTests(unittest.TestCase):
    def testMethod1(self):
        cut = TestCode()
        actual = cut.Method1()
        self.assertEqual(100, actual)

def teststaticInt(self):
self.assertEqual(1, TestCode.localInt)

    def testMethod2(self):
        cut = TestCode()
        cut.Method2(7, 4)
        self.assertEqual(28, cut.localInt)

if __name__ == '__main__':
unittest.main()

This example should illustrate that we can write some unit tests for our Python code fairly easily and quickly. It's nice to have a built-in framework for unit testing. There is also a "doctest" module available for doing more like system testing. I am pretty new to Python, so I haven't got examples of this as yet.

If anyone has any suggestions on tools or experience with integration with Visual Studio those comments would be appreciated.

Organizing Test Code

John E. Boal — Mon, 22 Sep 2008 17:21:35 GMT

Shawn Neal has a great post (ok it might be a rant) on organizing test code. I agree with his article, on the separation of different kinds of tests into separate assemblies that perhaps can be run at different times (not always with the build). Only the true unit tests should probably be running with each build, and for each developer. If there are other tests that run very fast (by my definition, a fast test either passes or fails in under 1/10 second [100ms]) they too could be included in per-build and per-developer runs, provided that they don't bog down the total test time too much, and provided that they are in separate assemblies.

Someday, when technology allows for it, we should even be able to run our tests in parallel threads or parallel processes (see the 2.5 target features for NUnit). Speed is truly important. If your development team has 10 developers, every minute they wait for a build to complete or a test run to complete is 10 minutes a day of development time wasted. If they build and test 6 times a day each (that's not very much BTW) that is one team development hour per day. Realistically it's likely to be more like 5 or 6 development hours a day.

So if someone is paying attention to how time is spent, we could say perhaps that spending a couple of development hours on the tests to make them run faster, and save one minute per execution would more than pay for itself in a single day.

The moral of this story is keep the tests fast, and when they aren't, fix them or move them out of the main test path.

Acceptance Criteria is Critical for Test Driven Development

John E. Boal — Sat, 20 Sep 2008 23:38:33 GMT

Acceptance criteria are the real key to being able to develop software in the agile model. Acceptance criteria is not really adequately represented by a use case, even with several alternate path scenarios... I have seen this and it just doesn't do the trick. A scenario is good, but it's really not acceptance criteria. We can however, analyze these scenarios and alternate paths, and usually come up with a pretty good set of criteria.

Beware of the incomplete criteria. It's darn hard to get it all. We know this. But, make an attempt anyway. Use the thinking that if it isn't in the criteria it won't be delivered, and that the developers can implement it any way they want... This thought should be scary enough for a customer that they will be forced to think hard about what they really want the software to do.

When we think we know what we are doing, we can write some tests, write some code. We think we have the idea, only to find out later that we had misunderstood or didn't have all the requirements. Acceptance criteria are absolutely the must-have thing for agile processes to succeed. We need the Customer or product owner [PO] to provide clear, concise, and specific criteria for our software.

I'm not saying that generating/gathering acceptance criteria is easy. It takes practice. It takes cooperation. It takes understanding on both the development team and the customer/PO that "if it isn't requested or specified in the acceptance criteria, it isn't going to be delivered." It's really kind of harsh actually to practice this, so I wouldn't advocate it. However, it might be a good idea to talk about it in those kind of terms while in the sprint planning or story negotiation phase.

Without the ability to know that we are doing the right thing, surely we will fail to deliver it. The customer knows exactly what they want. OK, most customers really don't know what they want. It is up to us as a development team to be able to know this, and help them out by asking lots of questions, and examining exactly what we propose to deliver. After all, that is why they hired us right?

Train your team. Let them know that the ancient role of "Business Analyst" is no place to be found in an agile team, and the whole team is really responsible for making sure to ask the right questions. I do like the whole user story concept, but it is far more practical in sprint planning to capture some of the actual acceptance criteria and store it attached to the story somehow, so that it is accessible by the team and can be implemented as automated tests.

Test automation is critical, mandatory, required, and not optional. Did I mention that test automation is required. Perhaps I should mention it again, in case my previous sentences weren't clear. Automate, Automate, Automate. Double-click run test suite, should be the demo in the sprint review and customer demonstration at the end of the sprint. This is really and truly the only way that agile teams can operate successfully.

If you have people doing any kind of manual testing (other than exploratory), this is a great opportunity to get the team to be far more productive by assisting the manual testers to get these tasks automated. Sometimes it is not practical to automate some manual tests. But in my experience - NOT OFTEN. If it looks hard to automate, someone should be asking why the code is so "untestable"... In most cases, the three or four dev or test hours it takes to automate the most difficult manual test case scenario is more than paid back in the number of times the automation can be run from then on. The number of bugs and breaks that it will prevent is a definite value add to the organization, and in my opinion, money in the bank.

September Agile Beer Night

John E. Boal — Fri, 12 Sep 2008 15:22:57 GMT

The next Agile Beer Night is coming up on Tuesday September 30, at the Celtic Bayou pub in Redmond from 5PM to 7PM. Be there or be square...

Sample Code for Selenium and WatiN tests

John E. Boal — Tue, 26 Aug 2008 15:03:11 GMT

I posted a simple sample of accepance test code in Selenium and WatiN along with a sample web site to test. You can download the zip file here.

I also have posted a Fitnesse fixture in that zip file that illustrates how we can create a simple test fixture for Fitnesse acceptance testing. The Fitnesse tests aren't in the set, but here is the page wiki code that makes use of the fixture:

!define COMMAND_PATTERN {%m %p}
!define TEST_RUNNER {dotnet\FitServer.exe}
!define PATH_SEPARATOR {;}
!path dotnet\*.dll

Here is an acceptance test using our BusinessObjectTestFixture test class:

!|FitnesseFixture.BusinessObjectTestFixture|
|UserId|Password|Authenticate()|
|administrator|secret0|ADMIN|
|admin|secret0|NONE|
|administrator|secret|NONE|
|user11|secret11|USER|
|user11|secret0|NONE|
|user|secret|NONE|

I also created some STIQ tests, here is the code for the tests and components. Extract this zip file under repository\ProjectRoot folder and it should be able to test the sample site also.

ATDDSTIQ.zip (3.57 KB)

List of Tools for Developers

John E. Boal — Tue, 05 Aug 2008 05:29:38 GMT

I have been asked recently about tools I prefer to use in my every-day development. Here is a list of tools, and where to get them.

Visual Studio 2008      Development IDE                   Microsoft
TestDriven.net         Test Runner                         TestDriven.net
WinMerge                  Diff / Merge tool                   WinMerge.org integrates with VS and Tortoise too!
nUnit                        Unit Testing Framework          nUnit.org     Also see mbUnit and Gallio
Selenium                   UI test framework               OpenQA       Also see WatiN
ReSharper                  Integrated toolkit for VS      JetBrains     (OK I don't actually use it but it's good.)
Tortoise SVN            Shell Integration with SVN      Tigris
Ankh SVN                  Visual Studio Plugin for SVN   CollabNet
Subversion (Server)     Version Control System          VisualSVN
Cruise Control.net    CI system                            ThoughtWorks
RhinoMocks                Mock Object System              Ayende
nAnt                         .NET Build Tool                     nAnt
Fitnesse                  Acceptance Test Tool            Fitnesse
STIQ                         Story Test Tool                    Solutions IQ
GIMP                         GNU Image Manipulation Prog. SourceForge
Notepad++               Smart Text Editor               SourceForge UK when Visual Studio just won't do...

Not development tools exactly, but extremely handy:
Process Explorer          Smarter Task Manager         SysInternals
FileZilla                      Upload/FTP client                  FileZilla
DivX                        Decoder                              DivX         Because sometimes we need to watch movies...

That's all I can think of at the moment, but am probably missing some things. I'm sure you'll all (please) chime in with what I forgot... :-)

Avoid Large Methods

John E. Boal — Mon, 04 Aug 2008 19:41:20 GMT

500 lines of code in one method? uh... U R DOING IT WRONG!

Long methods are a code smell. And a loud one at that. Methods should be focused on doing ONE thing. That thing can be simple or complex, but inner complexity should be refactored out into other (small) methods, keeping them each small and focused on a single task. A method with 4 nested levels of for/foreach/while statements is just so wrong in so many ways... Keep it simple. Keep it straightforward. Did I mention keep it simple?

Jeremy Miller posted a nice article a while back about short methods, and Ward Cunningham has another on C2 that Jeremy references.
We definitely need a Visual Studio plug-in to turn the background yellow when a method exceeds 20 lines, and turn red when it goes over 40 lines...

Here is a simple example. Code by intent means writing code by declaring what it does by well named (and usually long-named) methods. This method is only three lines long, but it conveys that there are two steps to be done, the second with the result of the first. This should be clear to anyone reading it. Further, the XML comment summary (missing from this example) should provide the reader with any information needed not conveyed with the title.

public void DoOneThing()
{
   int n = 0;
   n = DoStep1ReturnNumber(n);
   DoStep2WithResultOfStep1(n);
}

private void DoStep1ReturnNumber(int number)
{
   // simple method
}

private void DoStep2WithResultOfStep1(int number)
{
   // more simple stuff
}

Seriously, if there is something out there already that does this, please let me know! If not, I am going to have to write one...