Bad Layered Architecture

I recently discovered something I should have realized a long time ago -- Layers are less about modularity and more about isolation.


I've been working on a compiler that I'll describe as four different layers.  It could be depicted graphically as follows:

Here's a rough and simplified description of each layer:

1. Lexer and Parser - The lexer converts standard human readable text into tokens which are then processed by the parser.  The parser examines the tokens and makes sure they adhere to the specified context free grammar and builds an abstract syntax tree (AST) used for later phases.
2. Semantic Analysis - Examines the AST and confirms that variables are not used without being declared and that the operators and functions used exist for the specified types.
3. Type Checker - Verifies that types are used appropriately and as declared, e.g., that strings are not compared to integers.
4. Code Generator - Processes the AST and generates the appropriate machine code.

And there you have it -- a nice layered architecture, or so I thought. What I had really created was a layered mountain, one that required that I scale each prior layer in order to work with the next.  Each layer was its own module, but it was mildly coupled to the next, or worse, to the next and the previous layers.

Layering Is Not Enough

TDD alone didn't solve the problem.  In this case, it was quite easy to write tests, but the tests were basically acceptance tests and less unit tests.

Yeah, I thought layers were simple.  I made sure each layer was its own module. My code was even easy to test:

@Test
void generatedCodeContainsExpectedPattern()
{
    String input = /* ... */;
    MachineCode code = compiler.toCode(input);
    assertThat(code, behavesAsExpected());
}

But I had failed.  As time progressed, it became harder and harder to identify exactly where an error occurred.  For a while I justified myself, after all, it wasn't my fault I needed a compiler generator.  I eventually had to admit my failure.

Create Isolated Layers

I needed isolated layers, layers that permitted me to work independently of the others, layers that protected me from change, layers that could be examined individually.  The result didn't look much different, but was considerably easier to work with:

Between each layer that previously existed now stands an isolation layer with a single purpose -- to isolate behavior and functionality.

For example, the foundation of my my first code generator was a set of helper functions that were used to generate the correct code.  These helper functions have now been simplified and augmented with a builder class.  The builder allows me to test just the specific pieces of the code generation, without regard for my input. The helper functions now truly have a single responsibility. As long as the code generator calls the builder in the same way as my unit tests, I am now guaranteed that the generated machine code will be correct.

My newly inserted isolation code was dead simple and almost trivial to understand, yet I reaped huge paybacks.  Although I failed at first, in the end I learned something -- isolate my layers -- and that's a win.

Inexperienced Quality

When I first started programming I concentrated on one thing, making my program work.  Not only was that the only thing that I concentrated on, but it was the only thing I was taught.  My studies in computer science didn't prepare me to program, rather, they taught theory with an occasional programming project.

Knowing nothing about proper design or clean code, I slowly added more and more functionality to my program, I'd insert a few lines here and a few lines there until my functions became long and tangled.  Classes quickly bloated becoming god classes with an overabundance of  dependencies.

Over time, I learned more about design and started striving to always have clean code and follow the single responsibility principle.  I eventually had an epiphany -- a stronger developer refactors the current design into an architecture that supports the newly required features and improves the design's quality attributes; the weaker developer stays with the current design no matter how messy the eventual solution might become.

Quality code doesn't come overnight, grow with a degree, or sprout with certifications, it comes with experience.

Image from: Chris Dalrymple's Moblog

Don't Architect, Refactor

The test-driven development mantra is "red, green, refactor," but we far too often let other things creep into the process.  One of these things is domain knowledge.

Back in February, Gojko Adzik described his experience with Keith Braithwaite's "TDD as if you meant it" exercise. He was to TDD whether a stone in the game of Go could be taken, or not.  In the game of Go, each piece is placed on a grid and can be taken if it is surrounded by only one liberty  (i.e. free space) in any of the four cardinal directions.

His first test was to identify stones with two corners covered as having two liberties.  Yet, despite that, and armed with additional domain knowledge, his test started out like this:

GoGrid grid=new GoGrid(3,3);

See it?  Yeah, the first and only line.  It's a domain leak. And it's not just Gojko -- I do it all the time, inadvertently of course. Rather than starting off with the simplest thing that could possibly work, we have a tendency to inject domain knowledge into our code. Suddenly we find that we must create a few classes rather than a few short lines of code.  We've made our job harder and started forcing a design that didn't exist nor evolve from the code.  Our leaky domain knowledge introduces untested design, BDUF, but on a smaller, slightly less turgid scale.

With TDD,  the design is supposed to evolve, yet I often find myself saying, "this is easy to code, but...."  And the but kills me, it's usually something describing a design or architecture problem that I don't know how I'm going to solve, so I sit there thinking, and thinking, and thinking.

"Stop! Don't architect, refactor," I must tell myself. That is, I need to implement what I know, worrying very little about the design.  Writing the initial code isn't about having the best, most-pure design the first time.  The refactoring step is about cleaning up the design.  It's also been said this way, "Make it work. Make it right. Make it fast."