Active Software Specification (a.k.a. Test-Driven Development)

I used to ignore Test-Driven Development (TDD) for years. For some reason it sounded to me like “test development” which was not cool. Just recently I started suspecting that my view of how software products should be developed might be inline with TDD. So I took a formal class and not only was I happy to discover I had come up with the same concept independently but I also got my view clarified and taken forward.

The reason why I am writing this article is to share this knowledge with my fellow developers, because it has become evident from my interactions with other developers in my organization that while the term "test-driven development” is quite popular, its philosophy is largely misunderstood. My goal is to get the people involved in software development to rethink who should do what in the software development process.

Specification Driven Development

A typical software organization has its product development driven by some sort of a specification. That could be a text document, a prototype, a slideshow of screenshots, or a mix of those. Whatever form the specification takes, its function is to project what the product should look like. The main problem with this approach is that the specification is passive and it must be interpreted by humans. First, a developer must interpret the spec and implement his interpretation in source code. Then, the specification is interpreted by a tester who verifies the developer’s interpretation and implementation. Wherever those two interpretations differ, there is a bug filed, and wherever they don’t, it is assumed the specification is correctly implemented. Now we have two, more concrete, problems: 1) the organization churns on aligning interpretations, and 2) the fact a developer and a tester share the same interpretation doesn’t mean that matches the author’s intent.

Things get even worse with time. Towards the end of the release cycle, both the developer and the tester have a pretty good understanding of what the product should look like and specification authors tend to neglect keeping their specifications in sync with the products. The endgame is the time when the public surface endures many small tweaks that are too small to be worth updating the specification, but that are enough for somebody looking at the specification to consider it outdated. So when the next release cycle starts, the team has the dilemma: should they spend the time to put the specification in sync with the product, or should they focus on the incremental difference and specify only that? The latter option is much more attractive is is the typical winner. One more release cycle and the original specification will be forgotten for good. Along with the original spec is lost the ability to track use cases that may get broken broken.

Active Software Specification

Specification-driven development somewhat works for v1, but it gradually falls apart in subsequent releases. The main reason for that is the passiveness of the specification. The alternative would be to write the specification in a different form – one that need not be interpreted and that would not get out of sync with the product. One such form is a suite of tests. A test is not a projection of a product feature – it is an active verification of the feature’s implementation. Notice there is nothing to be interpreted any more.

More specifically, these tests a check-in tests, i.e. any attempt to check in a source code change would be rejected unless all those tests pass. Permissions to modify those tests should be restricted to the architect who authored them and a manager responsible for the final product. If this sounds too rigid, let me remind you this: a broken test means one thing – a broken use case. That should be approved by both a technical and a business owner. As long as these tests live along with the source code, they will remain in sync for many releases. That is the philosophy of Test-Driven Development.

Some Clarifications on Test-Driven Development

My observation is developers tend to avoid TDD due to a fear of “test first”. “Test first” is an assumption that all tests must be written before product development may start. In reality, some tests will indeed be written before any product code is checked in, and the more tests are in place the better. However, it is unrealistic to expect to have the complete test suite ready before any product development may start. Most likely tests will show up shortly before their corresponding product features.

The next controversy is who should write those check-in tests? My answer is: the same person who would otherwise write the specification. After all, these check-in tests specify the product. This doesn’t mean our industry has the tools to aid check-in test writers and reviewers to make a test suit look like a specification, but I hope will get there.

Then what would today’s software testers do? Let me clarify the meaning of this question: if a product satisfy all its check-in tests, then it is in full conformance with its architect’s intent; is there anything further to be verified? Yes, there is – the overall usability of the product, i.e. what applications can be implemented on top of this product (if the product is a component), or what operations can be performed using this (if this is an end-user product?) That is what today’s software testers should do. And I would call those people “application developers”. The result of the their work should be made available to customers either as free (but supported) utilities, or as commercial products.

Conclusion

I have authored specifications, and I have experienced the difficulty with keeping a specification current. I can point to a large number of outdated specifications, and I believe anybody can too. I continue searching for an alternative to specification-driven development. Regarding Test-Driven Development, it seems very promising - I have not practiced it as part of a whole organization, but I have adopted it in my own micro world and it works for me. What I find missing is tools to make a check-in test suite look like a specification, or the other way around – to extract tests from a textual specification (with extra visual content), but I remain optimistic textual specifications will start yielding to check-in tests.