Test-driven Development/Design when practiced properly, yields organic design growth as tests cause feature code to be written in response to failing tests. In *proper* practice, the entire code base is refactored as needed to ensure that all code and design smells are eliminated (while keeping tests passing). This essentially causes the design at any given time to integrate all the features written up to that point. Once this process is completed, the feature code should be relatively complete, if the tests were written well to dictate the behavior required by the customer. If Acceptance-Test Driven Development is used, the Acceptance Tests for the feature passing will indicate that it’s ready to ship. At any given point, the code base that is checked in should withstand the scrutiny of an outside code reviewer, and should embody all the best design principles. If it doesn’t, the “refactor” stage has not yet been completed.

Refactoring to Remove Code and Design Smells

I find many teams that either poorly implement or flat-out skip the step of refactoring the code base and overall code design as part of this cycle. This kind of organic development - sticking a new feature on a code base without integrating it into the design and making changes as required is like sticking a wad of gum on a bowling ball. It tends to be the entire code base lumpy, obtuse, and generally not pretty to look at. Teams can get away with this kind of development for a while more or less successfully (unfortunately). However this behavior causes LARGE problems for the team or those inheriting maintenance and new feature work on this code base. Often the team will have to stop to rewrite the whole code base at some point, because it’s too problematic to maintain the bowling ball and gum code. In practical business situations, it’s far far easier to maintain the overall design and integrate features into the design, course-correcting the design as needed to absorb all of the features into a coherent architecture.

This concept of going through the code base and refactoring all of it to embody all the features in it, and removing design and code smells does imply that if the design changes, so must the tests that correspond to the design. If the tests are tightly coupled to the code base, some design changes can be painful in terms of test maintenance overhead. There are methodologies that assist in decoupling the test from the code, and have the test just confirm the behavior without much knowledge of how the implementation is done. The use of polymorphism, mock objects to decouple behaviors from implementation, and the principle of dependency injection in general can assist with decoupling, and make test maintenance less painful (although perhaps somewhat tougher to write at the outset). There is overhead still however with design changes, and that should be expected, embraced, and *estimated* in planning…

Plan for Change

In iteration planning, changes to the design and implementation, along with test maintenance should be factored in to the estimates for work being done. When breaking out tasks for a story, I like to at least mention if not record a specific task for the design refactoring, and another for the test maintenance for design changes. I personally think these should be more or less estimable by most teams with at least rudimentary skills. If the team is new, add a task with a ballpark guess for the time, but be sure to add the tasks so they have visibility… Scoping the effort for the design changes and test maintenance will be easier and easier going forward as the team gains understanding of what impact new features have. And it is likely that for a team with a good solid design, these changes will be minimal if they are done incrementally.

Some teams prefer to finish the tasks for the features and their tests, and then have a single story for the design and test maintenance per iteration, and keep copying it from one iteration to the next. This works too, however I have seen issues where the iteration tasks don’t get completed, and the maintenance story gets bumped off or not completed in the sprint. This engineering debt is carried forward, and the code that is checked in for “completed” features for the sprint now carries debt with it as well.

In my experience, I have seen more success with making the elimination of design and code smells part of the “Done” criteria for each story, and in fact pre-check-in criteria. This way we can’t call a story “Done” [meaning completed, and shippable] until it *really* is. Design and test maintenance changes on a per-story level should be small, and usually bite-size enough for a team to be able to handle operating this way even in a short iteration period.

Don’t omit the design and test maintenance, plan for it specifically each iteration. This practice will help keep engineering debt from accruing from iteration to iteration. It is a key practice that will help keep a team, a project, and a code base (and a business) all on a successful course for the short, and the longer term.

When we are unit testing a piece of code, sometimes there are modules or even systems that we need to rely on outside the scope of the test. It is usually very difficult or expensive to spin up these large pieces just to test one small piece. Even attempting to do this can really slow down the execution of the tests, and that becomes a problem of its own. When possible, we'd like to separate the implementation from the caller's execution so that the control is really in the called module doing the work. This is the "Inversion of Control" or IoC principle, as it's often referred.

Dependency Injection is one mechanism where we can design our code in this way. Dependency injection [DI] is like "injecting" a small bit of test-controlled code into the main-line execution space while the test is running. The test can control the specific execution of the "dependency" code, and cause it to behave in a specific way so as to elicit a specific behavior from the mainline code under test.

My favorite mechanism to accomplish DI is to refactor code under test, extracting the logic of the dependency into a class that implements an interface. Then, we can create a test mechanism that implements the interface and behaves in the way that the test can control. We then replace the mainline "real" code with the test version at test time, bypassing the dependency and allowing us to remain focused on the code under test.
We can implement this with simple classes, or with Mock objects. The mock object frameworks out there today are quite robust and relatively easy to use, given an interface to implement for the dependency class. Rhino Mocks and Moq are some examples of frameworks that can be used for this type of test. Here is a simple example of some code under test:

using System;

namespace DIexample
{
    public class ClassUnderTest
    {
        internal IDependOnMe limit = new Limiter();

        public int HasDependencyOnExternalClass(int value)
        {
            if (value < 0)
            {
                return limit.LogicFunction(value);
            }
            else
            {
                return value;
            }
        }
    }

    interface IDependOnMe
    {
        int LogicFunction(int value);
    }

    public class Limiter : IDependOnMe
    {
        public int LogicFunction(int value)
        {
            if (value < -10)
            {
                return value;
            }
            else
            {
                return -10;
            }
        }
    }
}

At test time, we can replace the internal variable "limit" with a test-created instance of an object that implements the "DependOnMe" interface, and behaves in a specific way (like just returning the value perhaps). The variable "limit" is declared as "internal" scope, because we want the test code to be able to access it and replace its "real" object instance with a test-specific version. I keep the test assembly separate from the mainline code. However, I don't like the idea of making these mechanisms public unless they need to be, so instead in the mainline code, I use the internals visible to assembly attribute like this:

using System.Runtime.CompilerServices;

[assembly: InternalsVisibleTo("TestAssembly")]

This allows all "internal" scoped methods to be accessed by the test assembly, and keeps all the test code out of the mainline code, and maintains at least a level of security that being public doesn't offer.

Mock frameworks can be a big help in this type of DI testing. Rhino Mocks is here: http://ayende.com/projects/rhino-mocks.aspx it's a good free framework and fully featured. I've not used it but heard good things about MOQ: http://code.google.com/p/moq/

Remember to keep the execution isolated from the implementation whenever possible, and it will make unit testing much easier.

Most everyone is familiar with the C# construct of the conditional operator:

        (binary expr) ? true_result : false_result

However I've found that many people often overlook the null-coalescing operator ??. I see a lot of code that examines a string and assigns a value to it if it's null (like a default perhaps):

        string s;
...
        s = (title == null) ? "default" : title;

However, this can be written more simply using the ?? operator:

        s = title ?? "default";

This construct comes in handy when using nullable types and particularly converting them back into non-nullable types:

        int? i = null;
        int counter = i ?? 0;

Enjoy Programming in C#! (and don't forget: You got a test for that??)

What is "testable" code? How do I know if my code is "testable" or not? There is a lot of talk in agile teams about making sure we have testable code, so what exactly does that mean? I will try to explain here some of the things that make code testable.

TDD Methodology
If we are using test-driven development to write the code, it is inherently testable since the tests drove us to write it. However, it is still possible that over time, refactoring can be done in such a way to make it less so. This is not likely however since all the unit tests would be passing.

Dependencies
If the code requires so many external dependencies that we must essentially spin up large parts of the system to test it, this is a clear red flag for testability. We should be able to mock out dependencies and isolate the code under test. If we can't do that, it's not testable. Our dependency relationships should be reasonable and sensible. If there is a "smell" here it probably means that there's work to do to make it testable.

Internal States
If there are lots of internals and complex states without interfaces to access and manipulate them, this is another red flag that the code's not testable. If there is a lot of internal logic and things that happen based on state, this should all be unit tested. If there's not a unit test for each thing that goes on internally, it's not testable. Here again if code is written using TDD this usually doesn't become a problem.

Setup
If the code requires a large amount of setup code to test it, this again is another red flag. I look at the amount of effort it takes to validate that a class does the one thing it's supposed to do. If the test code is more than the main code, it's probably a sign that it's not very testable. Databases are notorious for this issue. I have some database code that's a good example of this concept. The amount of code and data needed to get the database to the state to exercise the few procedures there greatly exceed the code itself. This is kind of combining both the dependencies and internal state concepts, and I think it's definitely not very testable.

Testability is a goal we strive for, it helps us to be able to make sure the code operates successfully and is maintainable. Write good unit tests. Look for testability in the small, and that will help the overall system to be more testable as well.

There are many different kinds of bugs that we encounter in software development:

• missing functionality (doesn't do what's required)
• unanticipated/undesired behavior (it did what??)
  
• failure to check boundaries (or boundary conditions [e.g null])
• user-interface issues (usability)
• miscalculations (and logic errors)
• control flow errors (failing to break out of a loop, etc.)
  
• failure to handle errors (unhandled exceptions)
  
• security issues (Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, Elevation of privilege)
  
• data interpretation errors (what date is 11/12/13 anyway?)
  
• interoperability errors (sending/receiving incorrect messages or data)
  

and most likely lots of others. Each of these areas has bugs that are all examples of opportunities to write a test that were missed along the development path.

Here is a simple real-world example. Today my Log file writing routine threw an unhandled exception when the file it was trying to write to was in use by another application (this is failure to handle errors, above). This caused the app to crash with an unhandled exception. In truth, I had written the code TDD, so it had tests, but none of them held the file open while it attempted to log data.

This was an unanticipated condition, but probably one that should have been pretty obvious too. However, I was trying to keep the code as simple as possible, and it's not a multi-threaded app, so there wouldn't have been *internal* collisions anyway. Regardless, the app crashing because its log file is being edited is probably not the desired behavior. So (being the test driven developer) I wrote this test first, to illustrate the bug:

[Test]
public void TestLogFileInUse()
{
    using (FileStream stream = File.OpenWrite(Utilities.LogFileName))
    {
        Utilities.Log("test");
    }
}

it's really simple, but it definitely did fail when the Log() method threw an exception because his file was being held open by the test. Only after I had a failing test, did I then modify the code to catch the IOException and handle it without crashing the application. I could also check for an event log entry if the file write fails, if that was the desired behavior.

The test is fast and simple, and it now guarantees not only that I fixed the bug by making the test pass [not crashing, the only requirement here] but also that the issue will never come back again.

This may be an almost trivial case, but it does illustrate how to use TDD when fixing bugs. I shouldn't really ever be adding new functionality to the mainline code without a failing test of some kind. Refactoring doesn't count, because we only refactor when all the tests are passing. Refactoring doesn't add any new functionality, it is just reorganizing the existing code to make it better.

The worst thing about fixing bugs is when they pop back up later... We hate regressions, because then we have to re-do re-done work YET AGAIN... Lean says that defects are waste, and if they come back after being fixed, then that makes the time spent on fixing them the second time even more wasteful.

This method of "find a bug >> write a test" is a Test-Driven approach to solving issues and making sure they never come back. Let's make regressions an artifact of history, and never fix a bug without having a failing test first!

Test driven development is a great practice. But, sometimes we definitely should NOT use it.

right. who's this guy and where's John??

Really, sometimes the practice of TDD isn't in the best interest of the business. TDD has pro's and con's.

Pro's

• better quality code
• actually does what the developer wanted it to do
• can safely change/refactor without worrying if we broke it
  

Con's

• test code costs time and money to write
  
• test code is overhead, it has to be maintained in addition to the main line code
• some test code can be complex and harder to understand than the code it tests
  

The reason we test software is to mitigate risk. The risks we mitigate can be many, but we ought to have business value in mitigating a risk before we write a test. Some projects are so small and temporary that it doesn't justify writing tests. The business isn't interested in mitigating too much risk from the small project or temporary utility that isn't already covered by the standard things programmers do to run their programs before they call them finished.

Unit Testing is a great way to document functionality and illustrate that it works as intended. But the time and effort spent on writing the tests must be of value to the business - in the form of risk mitigation. For shipping products and tools that decisions will be based on, yes it makes sense to test it all thoroughly. The risk of customer problems and complaints, or bad decisions based on erroneous data are too high to leave un-mitigated. So we need to test them well to mitigate the risk and give that value to the business. Other times we can say a risk is just mitigated by the fact that we are "willing to accept" the risk. Acceptable Risk is a term I have heard used in threat modeling - indicating that the risk is so small or the problem so unlikely that we recognize it, but live with the risk that it may someday occur.

Testing is overhead, so make sure that the testing done is appropriate. Make sure that there is recognized and enumerable risk that can be mitigated by each test. Make sure that all the tests (unit, integration, functional, etc.) are warranted for the software being tested. I don't recommend under-testing either.

In Agile, one of our fundamental tenets is that we always strive to deliver value to the customer. For me, I strive to deliver value with each and every line of code, both main-line and in tests. Test the right things, and only at the right time.

TDD is a great practice to use for code success, as long as the *entire* process is followed...

write tests
write code
refactor
all tests green
we're done right?

Refactoring is the Key to Successful TDD.
We need to look at the changes we made as part of the whole, NOT just in the small. We may have just added a method to a class that makes the class now responsible for more than just the one thing it was before... That's now a code smell (or design smell). When we've refactored the code to make it testable, we're not really done...

We now need to take a step back, and look at the overall design, the archtecture, and how our implementation is satisfying the problems we are solving. We should make sure at this point in time that we can say everything is "well-designed" and "well-implemented."

So many teams that are new to TDD, or that just practice test-first development, tend to forget this CRUCIALLY IMPORTANT step... It's not done, it's not estimated, and often the consequences are simply ignored until it gets to the point that the entire code base is impossible to maintain. This Technical Debt or Engineering Debt goes unaccounted in so many cases. Sometimes, this result ends up being one of the factors that teams use to stop using TDD. When compared to traditional waterfall design models, the output of teams that omit this critical step does not nearly measure up to the designs of waterfall methodology.

There's no free lunch... We still need to have time in our plan for DESIGN. In waterfall, it's all up-front. With TDD and refactoring, we need to do it after we have written the code. We can't just "omit it" - otherwise we're not really "done."

Each time I complete a story, part of my done criteria is to hold up the entire system to a bright light, and look through it for code and design smells. If there are any, it really isn't Done Done Done. I may check in the working code, but I don't close the story as complete until this task is finished.

I estimate in a guess at least for time for sniffing out code and design smells on each story. My guideline for this is at least 10% of the time the rest of the tasks on the story take. Sometimes this clearly isn't enough, as when a story or feature causes significant re-design, or the adaption of an existing design to a new strategy. Use judgement on the estimate, but *AT LEAST* include the task for each story, even if it doesn't get estimated. This method will keep the issue out in the open and remind everyone that it still needs to be done.

If your product owner balks at this additional time (and they most always do) remind them that its a little penalty now, or a HUGE one later. Unless it's a very short-term project on a temporary system, this is almost always a required task, so that maintenance and upkeep - even in the next sprint - aren't blown out of proportion.

At the end of the day, the TEAM is responsible for deciding the best course of action to satifsy the the stakeholders, and even if the PO resists, the development management chain is *still* a stakeholder... As professional developers we really should have the ability to push back on a resistant PO with good quality engineering standards.

Educate your PO's. Make sure people understand how important this critical step is in the story. Place a task on each story for design refactoring. And, last but not least, you can always place a large poster of a nose with a red line through it in the team space and caption it *No Code Smells*" ...