I have been using IronPython lately, and am having a good time testing code with its built-in unit test framework. Here is a brief example of how to write some unit tests for IronPython (or CPython if that's your flavor). These unit tests leverage the "unittest" module available in Python 2.2 and above.

Here is a short test class:

class TestCode:
    localInt = 1
    @classmethod
    def Method1(self):
        return 100
    def Method2(self, a, b):
        self.localInt = a * b

Here are some sample unit tests for it, using the unittest framework:

import unittest
class UnitTests(unittest.TestCase):
    def testMethod1(self):
        cut = TestCode()
        actual = cut.Method1()
        self.assertEqual(100, actual)

    def teststaticInt(self):
        self.assertEqual(1, TestCode.localInt)

    def testMethod2(self):
        cut = TestCode()
        cut.Method2(7, 4)
        self.assertEqual(28, cut.localInt)

if __name__ == '__main__':
    unittest.main()

This example should illustrate that we can write some unit tests for our Python code fairly easily and quickly. It's nice to have a built-in framework for unit testing. There is also a "doctest" module available for doing more like system testing. I am pretty new to Python, so I haven't got examples of this as yet.

If anyone has any suggestions on tools or experience with integration with Visual Studio those comments would be appreciated.

Shawn Neal has a great post (ok it might be a rant) on organizing test code. I agree with his article, on the separation of different kinds of tests into separate assemblies that perhaps can be run at different times (not always with the build). Only the true unit tests should probably be running with each build, and for each developer. If there are other tests that run very fast (by my definition, a fast test either passes or fails in under 1/10 second [100ms]) they too could be included in per-build and per-developer runs, provided that they don't bog down the total test time too much, and provided that they are in separate assemblies.

Someday, when technology allows for it, we should even be able to run our tests in parallel threads or parallel processes (see the 2.5 target features for NUnit). Speed is truly important. If your development team has 10 developers, every minute they wait for a build to complete or a test run to complete is 10 minutes a day of development time wasted. If they build and test 6 times a day each (that's not very much BTW) that is one team development hour per day. Realistically it's likely to be more like 5 or 6 development hours a day.

So if someone is paying attention to how time is spent, we could say perhaps that spending a couple of development hours on the tests to make them run faster, and save one minute per execution would more than pay for itself in a single day.

The moral of this story is keep the tests fast, and when they aren't, fix them or move them out of the main test path.

Acceptance criteria are the real key to being able to develop software in the agile model. Acceptance criteria is not really adequately represented by a use case, even with several alternate path scenarios... I have seen this and it just doesn't do the trick. A scenario is good, but it's really not acceptance criteria. We can however, analyze these scenarios and alternate paths, and usually come up with a pretty good set of criteria.

Beware of the incomplete criteria. It's darn hard to get it all. We know this. But, make an attempt anyway. Use the thinking that if it isn't in the criteria it won't be delivered, and that the developers can implement it any way they want... This thought should be scary enough for a customer that they will be forced to think hard about what they really want the software to do.

When we think we know what we are doing, we can write some tests, write some code. We think we have the idea, only to find out later that we had misunderstood or didn't have all the requirements. Acceptance criteria are absolutely the must-have thing for agile processes to succeed. We need the Customer or product owner [PO] to provide clear, concise, and specific criteria for our software.

I'm not saying that generating/gathering acceptance criteria is easy. It takes practice. It takes cooperation. It takes understanding on both the development team and the customer/PO that "if it isn't requested or specified in the acceptance criteria, it isn't going to be delivered." It's really kind of harsh actually to practice this, so I wouldn't advocate it. However, it might be a good idea to talk about it in those kind of terms while in the sprint planning or story negotiation phase.

Without the ability to know that we are doing the right thing, surely we will fail to deliver it. The customer knows exactly what they want. OK, most customers really don't know what they want. It is up to us as a development team to be able to know this, and help them out by asking lots of questions, and examining exactly what we propose to deliver. After all, that is why they hired us right?

Train your team. Let them know that the ancient role of "Business Analyst" is no place to be found in an agile team, and the whole team is really responsible for making sure to ask the right questions. I do like the whole user story concept, but it is far more practical in sprint planning to capture some of the actual acceptance criteria and store it attached to the story somehow, so that it is accessible by the team and can be implemented as automated tests.

Test automation is critical, mandatory, required, and not optional. Did I mention that test automation is required. Perhaps I should mention it again, in case my previous sentences weren't clear. Automate, Automate, Automate. Double-click run test suite, should be the demo in the sprint review and customer demonstration at the end of the sprint. This is really and truly the only way that agile teams can operate successfully.

If you have people doing any kind of manual testing (other than exploratory), this is a great opportunity to get the team to be far more productive by assisting the manual testers to get these tasks automated. Sometimes it is not practical to automate some manual tests. But in my experience - NOT OFTEN. If it looks hard to automate, someone should be asking why the code is so "untestable"... In most cases, the three or four dev or test hours it takes to automate the most difficult manual test case scenario is more than paid back in the number of times the automation can be run from then on. The number of bugs and breaks that it will prevent is a definite value add to the organization, and in my opinion, money in the bank.

The next Agile Beer Night is coming up on Tuesday September 30, at the Celtic Bayou pub in Redmond from 5PM to 7PM. Be there or be square...