Test Driven Developer

Testable Code

John E. Boal — Mon, 15 Feb 2010 16:21:07 GMT

What is "testable" code? How do I know if my code is "testable" or not? There is a lot of talk in agile teams about making sure we have testable code, so what exactly does that mean? I will try to explain here some of the things that make code testable.

TDD Methodology
If we are using test-driven development to write the code, it is inherently testable since the tests drove us to write it. However, it is still possible that over time, refactoring can be done in such a way to make it less so. This is not likely however since all the unit tests would be passing.

Dependencies
If the code requires so many external dependencies that we must essentially spin up large parts of the system to test it, this is a clear red flag for testability. We should be able to mock out dependencies and isolate the code under test. If we can't do that, it's not testable. Our dependency relationships should be reasonable and sensible. If there is a "smell" here it probably means that there's work to do to make it testable.

Internal States
If there are lots of internals and complex states without interfaces to access and manipulate them, this is another red flag that the code's not testable. If there is a lot of internal logic and things that happen based on state, this should all be unit tested. If there's not a unit test for each thing that goes on internally, it's not testable. Here again if code is written using TDD this usually doesn't become a problem.

Setup
If the code requires a large amount of setup code to test it, this again is another red flag. I look at the amount of effort it takes to validate that a class does the one thing it's supposed to do. If the test code is more than the main code, it's probably a sign that it's not very testable. Databases are notorious for this issue. I have some database code that's a good example of this concept. The amount of code and data needed to get the database to the state to exercise the few procedures there greatly exceed the code itself. This is kind of combining both the dependencies and internal state concepts, and I think it's definitely not very testable.

Testability is a goal we strive for, it helps us to be able to make sure the code operates successfully and is maintainable. Write good unit tests. Look for testability in the small, and that will help the overall system to be more testable as well.

Find a BUG, Write A TEST

John E. Boal — Wed, 23 Sep 2009 03:24:00 GMT

There are many different kinds of bugs that we encounter in software development:

missing functionality (doesn't do what's required)
unanticipated/undesired behavior (it did what??)
failure to check boundaries (or boundary conditions [e.g null])
user-interface issues (usability)
miscalculations (and logic errors)
control flow errors (failing to break out of a loop, etc.)
failure to handle errors (unhandled exceptions)
security issues (Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, Elevation of privilege)
data interpretation errors (what date is 11/12/13 anyway?)
interoperability errors (sending/receiving incorrect messages or data)

and most likely lots of others. Each of these areas has bugs that are all examples of opportunities to write a test that were missed along the development path.

Here is a simple real-world example. Today my Log file writing routine threw an unhandled exception when the file it was trying to write to was in use by another application (this is failure to handle errors, above). This caused the app to crash with an unhandled exception. In truth, I had written the code TDD, so it had tests, but none of them held the file open while it attempted to log data.

This was an unanticipated condition, but probably one that should have been pretty obvious too. However, I was trying to keep the code as simple as possible, and it's not a multi-threaded app, so there wouldn't have been *internal* collisions anyway. Regardless, the app crashing because its log file is being edited is probably not the desired behavior. So (being the test driven developer) I wrote this test first, to illustrate the bug:

[Test]
public void TestLogFileInUse()
{
    using (FileStream stream = File.OpenWrite(Utilities.LogFileName))
    {
        Utilities.Log("test");
    }
}

it's really simple, but it definitely did fail when the Log() method threw an exception because his file was being held open by the test. Only after I had a failing test, did I then modify the code to catch the IOException and handle it without crashing the application. I could also check for an event log entry if the file write fails, if that was the desired behavior.

The test is fast and simple, and it now guarantees not only that I fixed the bug by making the test pass [not crashing, the only requirement here] but also that the issue will never come back again.

This may be an almost trivial case, but it does illustrate how to use TDD when fixing bugs. I shouldn't really ever be adding new functionality to the mainline code without a failing test of some kind. Refactoring doesn't count, because we only refactor when all the tests are passing. Refactoring doesn't add any new functionality, it is just reorganizing the existing code to make it better.

The worst thing about fixing bugs is when they pop back up later... We hate regressions, because then we have to re-do re-done work YET AGAIN... Lean says that defects are waste, and if they come back after being fixed, then that makes the time spent on fixing them the second time even more wasteful.

This method of "find a bug >> write a test" is a Test-Driven approach to solving issues and making sure they never come back. Let's make regressions an artifact of history, and never fix a bug without having a failing test first!

When NOT to use TDD

John E. Boal — Fri, 07 Aug 2009 05:11:13 GMT

Test driven development is a great practice. But, sometimes we definitely should NOT use it.

right. who's this guy and where's John??

Really, sometimes the practice of TDD isn't in the best interest of the business. TDD has pro's and con's.

Pro's

better quality code
actually does what the developer wanted it to do
can safely change/refactor without worrying if we broke it

Con's

test code costs time and money to write
test code is overhead, it has to be maintained in addition to the main line code
some test code can be complex and harder to understand than the code it tests

The reason we test software is to mitigate risk. The risks we mitigate can be many, but we ought to have business value in mitigating a risk before we write a test. Some projects are so small and temporary that it doesn't justify writing tests. The business isn't interested in mitigating too much risk from the small project or temporary utility that isn't already covered by the standard things programmers do to run their programs before they call them finished.

Unit Testing is a great way to document functionality and illustrate that it works as intended. But the time and effort spent on writing the tests must be of value to the business - in the form of risk mitigation. For shipping products and tools that decisions will be based on, yes it makes sense to test it all thoroughly. The risk of customer problems and complaints, or bad decisions based on erroneous data are too high to leave un-mitigated. So we need to test them well to mitigate the risk and give that value to the business. Other times we can say a risk is just mitigated by the fact that we are "willing to accept" the risk. Acceptable Risk is a term I have heard used in threat modeling - indicating that the risk is so small or the problem so unlikely that we recognize it, but live with the risk that it may someday occur.

Testing is overhead, so make sure that the testing done is appropriate. Make sure that there is recognized and enumerable risk that can be mitigated by each test. Make sure that all the tests (unit, integration, functional, etc.) are warranted for the software being tested. I don't recommend under-testing either.

In Agile, one of our fundamental tenets is that we always strive to deliver value to the customer. For me, I strive to deliver value with each and every line of code, both main-line and in tests. Test the right things, and only at the right time.

Red, Green, Done.

John E. Boal — Thu, 04 Jun 2009 20:17:50 GMT

TDD is a great practice to use for code success, as long as the *entire* process is followed...

write tests
write code
refactor
all tests green
we're done right?

Refactoring is the Key to Successful TDD.
We need to look at the changes we made as part of the whole, NOT just in the small. We may have just added a method to a class that makes the class now responsible for more than just the one thing it was before... That's now a code smell (or design smell). When we've refactored the code to make it testable, we're not really done...

We now need to take a step back, and look at the overall design, the archtecture, and how our implementation is satisfying the problems we are solving. We should make sure at this point in time that we can say everything is "well-designed" and "well-implemented."

So many teams that are new to TDD, or that just practice test-first development, tend to forget this CRUCIALLY IMPORTANT step... It's not done, it's not estimated, and often the consequences are simply ignored until it gets to the point that the entire code base is impossible to maintain. This Technical Debt or Engineering Debt goes unaccounted in so many cases. Sometimes, this result ends up being one of the factors that teams use to stop using TDD. When compared to traditional waterfall design models, the output of teams that omit this critical step does not nearly measure up to the designs of waterfall methodology.

There's no free lunch... We still need to have time in our plan for DESIGN. In waterfall, it's all up-front. With TDD and refactoring, we need to do it after we have written the code. We can't just "omit it" - otherwise we're not really "done."

Each time I complete a story, part of my done criteria is to hold up the entire system to a bright light, and look through it for code and design smells. If there are any, it really isn't Done Done Done. I may check in the working code, but I don't close the story as complete until this task is finished.

I estimate in a guess at least for time for sniffing out code and design smells on each story. My guideline for this is at least 10% of the time the rest of the tasks on the story take. Sometimes this clearly isn't enough, as when a story or feature causes significant re-design, or the adaption of an existing design to a new strategy. Use judgement on the estimate, but *AT LEAST* include the task for each story, even if it doesn't get estimated. This method will keep the issue out in the open and remind everyone that it still needs to be done.

If your product owner balks at this additional time (and they most always do) remind them that its a little penalty now, or a HUGE one later. Unless it's a very short-term project on a temporary system, this is almost always a required task, so that maintenance and upkeep - even in the next sprint - aren't blown out of proportion.

At the end of the day, the TEAM is responsible for deciding the best course of action to satifsy the the stakeholders, and even if the PO resists, the development management chain is *still* a stakeholder... As professional developers we really should have the ability to push back on a resistant PO with good quality engineering standards.

Educate your PO's. Make sure people understand how important this critical step is in the story. Place a task on each story for design refactoring. And, last but not least, you can always place a large poster of a nose with a red line through it in the team space and caption it *No Code Smells*" ...

SQL Server Queues PRINT output

John E. Boal — Wed, 06 May 2009 17:00:43 GMT

Ideally, SQL queries are executed really fast (sub-second). However there are times when it takes many seconds, even many minutes to execute each step. We have traditionally used the PRINT statement in SQL to output trace type information to the console so we can see what's happening.

Recently I noticed that the PRINT statement output wasn't coming out when I expected it should. In doing some research on the issue, I discovered that the PRINT statement output is queued and batched rather than real-time.

The solution for this problem is to use the RAISEERROR statement instead. Here is an example:

DECLARE @msg NVARCHAR(MAX) = 'status message'
RAISERROR (@msg, 0, 1) WITH NOWAIT

The severity of 0 tells SQL that everything is still fine, and NOWAIT instruction tells SQL not to queue it but to output immediately. The 1 is a locator that can be used to tell someone where to find the source of a particular message, if they are kept unique throughout the code.

So, for long-running queries, use RAISEERROR and get those status messages out in real-time!

< Rant >

John E. Boal — Wed, 08 Apr 2009 19:05:23 GMT

OK people. STOP putting cowboy code on my desk. Seriously. TEST your software. REALLY. Please STOP writing BRAND NEW LEGACY code.
< /Rant >

OK I feel better. But not really. Just because a system is not considered to be a "production release" (meaning that people outside the particular organization or business won't have access to it) it DOES NOT mean that we can cowboy it up and plop a steaming pile of untested code out there for our colleagues or coworkers to consume.

ALL CODE NEEDS TESTS. Period.

Untested code is NOT finished. Please don't just assume that if the code "works" that it's ready to inflict on other people (or even yourself). Self-inflicted legacy code casualties are the second highest cause of career paralysis, according to 4 out of 5 dentists. By definition, code is "legacy" if it doesn't have tests.

Here's a plan to get your tests caught up.

Start Small
At least we need a set of "sanity" checks... if nothing else. Create a "test suite." Use xUnit or the like, keep it simple. Start with the checks for the obvious output... if that doesn't show up, then something is clearly broken.

Add Some More
One at a time if you have to, add another test for something. Any kind of test will do... unit, functional, acceptance... I recommend starting with acceptance and working backward. We already blew the op to be able to do TDD and get unit testing done, so now let's just make sure the surface behavior is adequate. Definitely do go in deeper if time permits.

New Feature
Each time a new feature is added, update the test suite to make sure everything is happy with changes that ripple out from the effect of the feature. Again - at least make sure that the obvious functionality is tested on the new code. Don't slack on it, keep at least the new code from being legacy.

Lather, Rinse, Repeat
Pick a time each week (say 4:30PM on Wednesdays) to put in a half-hour on writing one new test. Schedule it. Block the time out on your calendar so someone can't schedule a meeting... And hide if you have to - to avoid distractions.

What to Write
Focus on happy-path code tests only if there are none for that feature. Otherwise, do alternate and failure path tests, so that the behavior of the system is understood in circumstances other than the best-case. 90% of bugs occur in the not-happy path.

Using this strategy will get testing started, and contribute to a more robust and reliable product, no matter what it is. Remember that most "side projects" or "utilities" that are developed in-house will be used by your own people. We want them to have the best.

"But it's just a spreadsheet." OK, yes it is. I say we need to make sure to test even the lowly spreadsheet. Because, next week someone in Finance is going to discover it and begin to use it for their payroll planning... if there is a critical flaw in it, it might be YOUR paycheck that gets eliminated!

We should realize that even internal tools and web sites are usually used to make business decisions of some kind. We want to make sure that they are made using reliable tools, to the extent possible. Business relies on its data, now more than ever. Competition is fierce in a down economy - even small mistakes can be a big problem.

"Bread's not done until it's baked" - and neither is code! SO PLEASE: Test it. Bake it. Slice it... and don't get burned!

Agile Developer Series - Continuous Integration

John E. Boal — Tue, 10 Feb 2009 19:38:47 GMT

Here is a presentation I wrote to explain why Continuous Integration is important, and touch on some of the philosophy of how we think of CI. There are a lot of tools and techniques for CI however I did not go into specifically in this presentation. Scenario Examples may be available in the future as scripts for specific tools to build software with CI. If you want to see these, please reply and let me know.

PowerPoint version:

AgileDeveloperSeries-ContinuousIntegration.pptx (79.86 KB)

PDF version:
AgileDeveloperSeries-ContinuousIntegration.pdf (650.31 KB)

What Makes a Successful Agile Developer?

John E. Boal — Sun, 01 Feb 2009 04:19:49 GMT

What makes a successful Agile developer? How are Agile developers different from regular developers? Here is a short presentation and video on the topic.

Video (xvid) 40MB

Presentation (PDF) 430KB

Writing Unit Tests With Test Driven Development

John E. Boal — Tue, 20 Jan 2009 01:11:28 GMT

I recently recorded a tutorial video of a presentation I am working on. The video is intended to be an introduction to TDD, and how actually to go about writing tests and code using TDD. I hear a lot of people using the term "TDD" without really understanding it. They typically are referring to "unit testing" or sometimes even test-first development, neither of which are really TDD. So my thought was to show it actually being done.

It is my first attempt ever to record a video presentation, and there is some kind of hum on the audio I couldn't get rid of - sorry. I was able to convert it to a down-res format (quick-time), using the HandBrake converter tool. Hopefully it's still readable. This is unscripted and off-the-cuff, so I am sure there's lots of room for improvement. If you have suggestions, please feel free to post a comment below. Otherwise, enjoy!

Writing Unit Tests Using Test-Driven Development Apple QuickTime format, 65MB, 28 min.

Scrum, TDD and Seven Other Buzzwords

John E. Boal — Wed, 17 Dec 2008 02:33:12 GMT

"We need to do TDD" said the client.
"Our team is not producing quality code, and aren't performing up to our expectations."
"So, we need to introduce TDD to make sure we get what we need."

Uh-huh I thought. Thinking to myself... so, why aren't your developers producing quality code? What are they doing instead? Are they even unit testing at all now? Do they know how to test or are unit tests just an afterthought? oh yeah, and are there any other methodologies you need to add for your team to be buzzword-compliant!?

TDD is not just a buzzword. TDD is not just a methodology a team can adopt if they are in trouble. Not hardly. TDD is the top of the pyramid. It requires a mind-shift that is not tremendously difficult, but at the same time is very difficult to accomplish without discipline. Not every team is ready for this step. I would hope to have a cohesive, collaborative, and performant team, teach them how to write good tests (first or last), show them how acceptance criteria can be automated into tests that guide the team toward completion. After going through all of these pieces, I would then introduce TDD as a small thing, and the discipline required would probably already be there at that point.

While it may be in vogue to say a team is "agile" and uses "tdd" (note: here in lower case) it's much more important that a team is producing quality software - buzzwords or not. I am not exactly advocating non-tdd practices, but on the other hand we need to be practical about delivery also. If a team can truly do test-driven development, then congratulations I say, they are well positioned to be at the top of the heap. If not, I would try to lead the team in the direction to be ready for TDD, whether or not it gets used. Strong unit testing along with other kinds of testing will deliver the quality that customers need. So, evolve. Learn, Test well. And, if you can, use TDD.. it's great!