Test-Driven Development Justified

I stood in shock, in utter awe and amazement.  I was talking with one of the lead programmers of an open source project, with whom I'd been in discussions countless times over the last few years.  He was a programmer who commanded my respect, and the respect of everybody else in our IRC community.  He wrote software that was virtually bug free and did exactly what it was supposed to.   He used the idioms of the language, appropriate patterns familiar to capable programmers, and consistently demonstrated a sound understanding of theory vetted with practicality.  And yet, despite all of this, he didn't know what TDD was.  To be fair to him, I bet he did, but he didn't recognize the acronym and asked me what it meant.

The TDD Tax

Misko Hevery recently mentioned a tax associated with testing.  I personally read that as the tax associated with TDD, which may or may not be exactly what was intended.  What exactly is that tax and how can we separate the necessary part of the tax from the unnecessary?


Let me frame the details of the tax in light of a commercial project that has essentially taken 1.5 years, full time.

Project Total	Lines	Chars	Chars /Line
All Source	75478	1939605	27.02
Unit Tests
Backend	2,116	52,755	24.93
Model	5,972	168,279	28.18
Main	1,004	23,302	23.21
Utils	4,178	102,358	24.50
Widgets	8,747	251,869	28.79
Core	702	16,256	23.16
Support	157	3,272	20.84
Total	22,876	618,091	27.02
Project Total	75478	1939605	25.70

The statistics above describe a desktop application written with Qt.  Unit tests were written with Qt's QTestlib which allows almost all interactions with widgets to be tested programatically.  Qt supports a model-view architecture which also helped test the GUI components. The application consists of about 75 thousand lines of source code and 23 thousand lines of tests.  The project actually should have more tests, but we weren't diligent enough at the time (and we regret it now).

Typing Tax


The first part of the TDD tax is the required extra typing.  I'd like to believe that everybody knows that being the fastest typist in the world does not make them an uber-efficient, super effective programmer.  Rather, typing speed has very little impact on the quality or effectiveness of a programmer.


How much is the typing tax? I type about 100 words per minute (WPM) for standard text, so let's assume that I can only type 60 when programming, because of all the extra special characters.  Although WPM has different definitions, I'm going to use five characters as a word.  Thus, 618,091 chars / 5 chars/word gives me 123618.2 words.  And 123,618.2 words / 60 WPM = 2060.3 minutes.  And 2060.3 minutes / 60 minute/hour gives 34.34 hours.


Yes, only 34.34 hours.  I could type the entirety of my unit tests in only 34 hours... in less than one forty hour work week.  In table form:

Words @ 5 chars/word 123618.2 Minutes @ 60 WPM 2060.3 Total Hours: 34.34

Unfortunately, I didn't keep track of time writing the tests, so I don't know what percentage of the time I spent writing tests and what percentage of the time I was developing production code.  But, for sake of this blog post, I'm going to assume 10% of my total development time was dedicated to writing tests.


That means that I spent about (1.5 * 52) * 40 / 10 = 312 hours writing tests.  Of that time spent writing tests, 34 hours, or 9.1% was spent typing.  The other 277 hours or 90.9% was testing related.

Breaking it Down Further

Where else was that 277 hours spent? Some of it was inevitably spent trying to figure out how I could test a certain part of the application.  Should that be part of the TDD tax?  Perhaps, but if I'm striving for quality, I am still going to test it somehow, aren't I?

In some cases it will be easy to test manually and difficult to test programatically, such as with unit tests.  In these cases, it could be said that I'm paying a tax.

I'm not sure how else I might be spending that time.  I imagine some of that is spent deriving and discovering yet unknown requirements.  Time is likely also spent working on some design related activities -- trying to discover how different pieces of the application can best fit together.  Another small portion of that time will be spent running the unit tests time and time again.

ROI


What I'm really doing, throughout this process, however, is making a deposit against the need to run the tests multiple times.  The more times I run the test, the more payment I get back; it compounds, like interest.

A bunch of different numbers wouldn't mean much here.  For a one off project, or a project whose requirements are truly known up front, perhaps the tax associated with TDD isn't worth it.  On most real-world projects, however, requirements will change... a lot.  And each time a requirement changes or part of the application is re-written, the automated tests pay interest.  I can change, replace, or re-write large chunks of functionality and know with certainty that I haven't changed any important behavior.

If you haven't tried paying the TDD tax, you should; you might find you'll later need to collect some of its interest.  When you've diligently attempted TDD, spread the word for many still don't know about it. Even if it doesn't work for your situation, it might work for theirs.