Overwhelming Complexity

Joel Spolsky recently wrote about how he "was reminded of why student projects, while laudatory, frequently fail to deliver anything useful," which made me reflect on some of my experiences as a student of computer science at Eastern Washington University.

I was lucky enough to take a number of independent study classes wherein I was able to "study" at my own pace while meeting regularly with my instructor.  In one such class I was studying compiler design. Because of my love for programming, I essentially made it into a project class and agreed with my instructor to work through a number of chapters through the year.

Introducing Complexity

Although the first quarter went well, my progress quickly slowed in the second quarter.  I was diligent in my use of time, so that wasn't the problem.  I was reading through the material, and understanding the material, so that wasn't the problem.  The problem was the complexity.

By the time I had all the basics of the compiler in place -- the lexer, parser, semantic analysis, and tree building processes setup, the complexity started to kill me.  I no longer knew exactly where I should add the various pieces of functionality.  My naming was sufficiently poor that I'd get confused about which piece did what; I'd sit there trying to figure out what line of code I could tweak, what class I could create, what design pattern I could use.  I was attempting to take time to design it correctly, but didn't refactor to a reasonable design, because I didn't know how.  The complexity became overwhelming.

Overcoming Complexity

Many projects fall into the trap of complexity.  But in all but the worst of cases, there's a way out -- one line of code at a time.  Michael Feather's Working Effectively with Legacy Code details steps that can be taken to alleviate the complexity buildup over time, but for greenfield projects TDD is a boon and plays a huge role in reducing complexity.

How complex can a project get if you can test and isolate all the pieces and each piece handles a single responsibility?  Yes, software projects can still become complicated, but they should still be manageable and understandable. TDD helps keep the software organized, layered, managed, and testable. I highly recommend it.