Tuesday, April 01, 2008
Acceptance Test Driven Development, Explained

Acceptance criteria should be our measuring stick when we write software. These criteria should give us the direction to go in writing features, and ultimately the final answer on whether the software is "done" or meets the customer's need. When the customer (or the customer representative, product owner, or business analyst) gives us a notion of what the software is to do for them, it represents a guideline and a goal for our development.

  This article is a discussion of ATDD in an Agile development context with Scrum project management and Extreme Programming [XP] development methodology. However it does not apply specifically to Agile. These same techniques can be applied with almost any mechanism.

  Test-Driven Development [TDD] says that we should first write a test before we write the code. This forces us to understand clearly what the code is to do, because we have to write the test to verify it. Writing the test is usually the hard part - the code next is easier. This is where the discipline of TDD is most needed. The test itself should be failing when executed, since we don't have the code under test [CUT]. This is a good sign - we need to actually "test the test." We then write as little CUT code as needed, just to make the test pass. Then, we start all over again - write a test, some more code, etc. We refactor the CUT as needed, to make it a cohesive, maintainable design, while still keeping all the tests passing. In this way, we have a complete test framework wrapped around our high-quality code, which makes certain that the code always functions the way we intended no matter what kinds of changes we make internally to it.

  This process of TDD as described, is the traditional mechanism we now refer to at Unit Test-Driven Development, or UTDD. There are higher-level tests that we can write also, that have the same impact on our development process. Applying this same concept of a failing test driving us to write code is where this discussion now leads, but at one level of abstraction higher. We now look to the Acceptance Criteria to drive our TDD, or now ATDD.

  1. Acceptance Criteria gives us a target for writing Automated Acceptance Tests [AAT's] a. Acceptance Criteria need to be written into an executable set of automated tests. b. AAT's should be able demonstrate to the customer that their criteria for the functionality has been met. c. Ideally, this should include as much detail as the requirements bear out, and as many tests and checks are necessary to convey that each of the criteria has been accounted for.

  2. AAT's give us a reason to write code (they are failing - no CUT yet) a. AAT's can be written before writing the code, or written one at a time following this process.

  3. We now write the empty framework for the CUT. Some AAT's may still be failing, but that's OK. a. This step deviates from strict TDD in that we are writing code, but not for the intent of passing a test. b. One example is that if a criteria states that a customer is able to bring up a web page with a set of information, obviously we need to provide a web page. c. This framework concept is creating the empty page or container for the code we are going to write. d. The empty framework should not cause AAT's to pass, because AAT's should written to test at a higher level of functionality.

  4. To make the AAT's pass, we need to start providing the functionality inside the empty framework. Here we need to apply the standard TDD concepts, and write a failing unit test.

  5. We now have a failing unit test, so we write enough CUT to pass that test.

  6. We repeat at step 4, refactor, and continue following the UTDD process as described above.

  7. When the functionality that the AAT's require is there, they will be passing, Unit Tests [UT's] are passing, and our story (feature) is complete. We now continue to step 1 again for the next story.

  How do I do it? What do I need to get started?

Acceptance Test Driven Development is a practice that can yield good returns on the time invested. Customers will be pleased with the software they receive, bugs will be fewer, and delivery times will be shorter. However, there are a few things that are needed to make this practice effective in an organization.
  1. Buy-in
    1. People must be bought-in to the concept that test driven development is a good thing.
    2. Management must support this practice, for if not, it will be a much more difficult thing to accomplish.
    3. Developers must be willing to put aside their "classical" training and try something new.
  2. Testing Tools
    1. If we can't write an automated acceptance test for our product, we can't do ATDD. We need a tool designed for testing the particular type of product we are shipping.
    2. Story Test IQ [STIQ] is a great tool for automated testing of web applications. Straight Selenium RC under C# code, driven by NUnit also works great, in my experience. VSTS has some good tools also, as do Fiddler, and probably lots of other products.
    3. The framework must be in place on day 1 of the iteration to begin writing acceptance tests. If it isn't available, we won't be able to do ATDD.
  3. Skills
    1. Developers need the skills to be able to envision what is to be delivered, and be able to write automated test scenarios for the acceptance criteria.
    2. Testers need to be able to validate that the tests cover enough of the functionality through testing the criteria, and provide other functional, integration, load, and performance tests in addition to the developers' unit tests and acceptance tests.
    3. Management needs to be able to direct the right resources at tasks, no matter whether they are Development or Test tasks. Sometimes it's best to blur the lines between the disciplines to make this practice most effective.
    4. Customers need to be aware that they are actually going to get what they specify (and perhaps ONLY that much). Education on criteria evolution is almost always required for customers. Don't be surprised if only 50% of the criteria are discovered in sprint planning and story generation for the first couple of iterations.
  4. Experience
    1. TDD takes discipline, and ATDD even more so. This is not a practice for an organization just beginning down the road of TDD. Put in at least 6 iterations of using strict TDD first before trying ATDD. The experience gained with using TDD will naturally flow into ATDD, and the organization and the customers will see the benefits.
  5. Perseverance
    1. Good things don't happen instantly. It takes some time to see the benefits of ATDD.
    2. Be patient for a few sprints. It should take approximately three iterations to get it dialed in.
  6. Customer Cooperation
    1. Customers must be available for providing their criteria for acceptance.
    2. Customers generally are not experts in this type of "specification" of criteria, and must often be helped through the process at first, to generate usable criteria.

ATDD | TDD | Testing
Tuesday, April 01, 2008 10:10:03 PM (Pacific Standard Time, UTC-08:00)  #    Comments [0]  |  Trackback
© Copyright 2013, John E. Boal