Adam @ Bitscribe

Let’s imagine that you had never before tried to open any canned food. Someone hands you a canopener. What do you think of this device?

It appears completely pointless. You can speculate on what it might be useful for, but mostly it will just seem like a contrivance of gears, blades, and levers without purpose or meaning. Probably you won’t even be able to remember the details of what it looks like if asked a day later to describe it.

Now further imagine you are handed a can of food and asked to open it. After variously trying to twist off the top, cut it open with a knife, and beat it against the countertop to open it, you give up in frustration. Then someone hands you the canopener - the same one you saw yesterday - and some basic instruction on its use (perhaps just a drawing of the right position to hold it in).

A light goes on in your head. The formerly pointless contrivance transforms into a magic wand that allows you easy, neat access to the delictable contents of the can. After using it you marvel at the clever contraption. Asked the next day to describe the can opener, you breathlessly go on and on about how this amazing series of gears and circular blades can be used to perform a most impressive feat.

The canopener prior to trying to open the can was a solution without a problem. Meaningless, arbitrary, an apparent waste of materials and complex design. But once you’ve grappled with the problem that the tool is designed to solve, you gain a unique insight and appreciation into the tool’s usefulness.

And so my clumsy metaphor closes, leading us to my true point: software tools. If you’ve never written a program that needed to manipulate data records with hundreds of thousands of entries, a SQL database seems like bloated, overly complex, contrived tool. If your web applications have always been limited to a few short PHP scripts, objects and model-view-controller frameworks probably look like nothing more than huge amounts of red tape.

Here’s the interesting bit: these observations are actually true. Canopeners without cans are just some much useless metal and plastic. SQL is not worth all the necessary infrastructure and specialized knoweldge for a program which is going to store a dozen phone numbers. And MVC on a simple web script is a hinderance rather than a help.

The solutions-without-problems concept is good to keep in your head for two reasons. One, when you show a useful tool to someone and they don’t get it, chances are they just have not grappled with the right problem yet. If you wish to enlighten them, then what you want to do is show them the problem instead of the tool. Once they have fully come to grips with the problem, it should only take a casual inspection of the tool to see its value.

In a parallel fashion, when you read or otherwise learn about a new tool and it seems pointless, then you should assume that haven’t come face-to-face with the problem yet. To understand the tool you’ll need to find them problem first, then get more details on the tool.

The worst offender of solutions-without-problems is academia. In high school I failed to grasp trigonometry in any serious way because I couldn’t see what possible benefit it was to compute the sine or cosine of an angle. Later on when I began writing programs to calculate geometry in 3D games, the usefulness of trig dawned on me like a revelation. “Why the hell didn’t my teachers tell me that this was actually _useful_?” I thought. Well, they probably tried, but without a (real) problem to solve, sine and cosine were just arbitrary functions.

I see this pattern heavily in CS courses: for example, teaching object-oriented programming on a small, one-person project where the benefits of OO are nearly non-existent. (In fact the extra overhead imposed in terms of writing header files probably makes it a worse alternative to procedural code.)

How can we get away from this approach to learning? I like to take what I call the Throw Them To The Wolves approach. Also known as the School of Hard Knocks. Throw yourself at a hard problem, let yourself get intellectually torn to bits, and then retreat to seek possible tools designed for the problem.

Catapult is a side project I’ve been working on this past few weekends, with the goal of bringing a command-line style interface to web navigation.

Think about how one normally navigates to the web apps one uses every day. If I want to compose a new post on my blog, for example, I go to the blog homepage (by entering in the complete url, maybe utilizing my browser’s autocompletion), then grab my mouse and click on the admin link, then click on the write link, then switch back to the keyboard to start typing. This is no big deal for just one app, but think of all the actions like this we perform every day. Web searches, Gmail, Yahoo yellow pages, Google Maps, Craigslist, eBay, your bank’s website, and so on. Typically we do all this with bookmarks, urls typed in from memory, and lots of clicking on navigation links. It’s not very efficient.

Catapult lets you create commands that act as shortcuts, in some cases dynamic ones. So for example a Google search would be “g [search terms]”. I can launch Gmail with “gmail”, but I can also jump straight to the compose window with a recipient already entered by typing “gmail someone@somewhere.net”. Searching my inbox is just as easy: “gmail /thing to search for”.

It’s interesting to note that using the Gmail launcher has increased my productivity in an unexpected way. I often need to send an email about something, but opening my inbox and seeing some new messages that have come in distracts me from that task. By jumping straight to the compose window I don’t even get a chance to see the inbox and thus avoid unintended distraction.

Check it out, and post any feedback here; or drop me an email with your comments.

The (Misguided?) Faith in Unit Tests stands as a rare counterpoint to the flurry of pro-TDDers (one of whom is me). The author makes some great points; but they have more to do with incorrect applications of TDD than flaws in the approach itself.

Let’s separate the technical design of software applications into two layers: high-level, which is an outline of all the working parts and how they fit together; and low-level, which is a description of the details of each part. A high-level design of a shopping cart, for example, might be something like: “Products represent items the user can buy, and belong to one Category. Cart is a container for CartItems, each of which specifies a quantity and is linked to a Product.” Design decisions you would make at this level are things like: can a Product can belong to more than one Category? Or: is multicurrency implemented through its own class, or just a currency_type enumerated field alongside any price field on an object?

The low-level design is a description of the interface to each object. For example, how does one add items to the cart: cart.addItem(item, or cart.getItems().add(item), or item.addToCart(cart)? Do you copy the price of the Product onto the CartItem when it is added to the cart, to prevent updates to the Product from affecting the customer’s checkout price? Does the shipping calculator connect to the shipping carrier’s API every time it needs to calculate a cart’s shipping price, or does it cache a local table of weights and shipping values to avoid slowdown from the network cost of the transaction? Most of these decisions affect either the internal implementation of the objects, or the APIs they export.

Tests, and unit tests in particular, help somewhat with low-level design, but not at all with high-level design. The latter is informed by the needs of the application. High-level design defines the mapping between software and the real world. Tests have a limited scope: their whole universe is the application, and anything beyond the boundaries of the code cannot be addressed. (It should be noted that business logic tests are one attempt to bridge this gap; we use this for some of our apps, but it isn’t appropriate in all situations.)

Furthermore, TDD is not so much a way to help create your design as a way to enforce it. Yes, writing the unit test first is a good way to discipline yourself to create a clean, straightforward, and orthogonal API for any given class, and I make this very point repeatedly in my screencast on test-driven development. But this is only half the story, maybe less. TDD’s real benefit over the long term is documenting and enforcing the interface and functionality of an object. A unit test says: “This is how the object is supposed to work, and I can prove it: just run me.”

So if someone new is working on the project, they can find out how multicurrency support is handled or what you have to do to add an item to a shopping cart by looking at the tests. Better yet, if they change an object’s implementation in a way that is inconsistent with the low-level design - for example, tinkering with the pricing code for products without properly handling multicurrency - this will break the tests, thus forcing the developer to confront the fact that they have a misperception about the design.

Where is the right place for the high-level design to live? For this I don’t think we have any solution other than traditional documentation. To some degree a good UI communicates many aspects of the design: for example, a breadcrumb trail shows the relationship between the primary objects (e.g. Store > Category > Product). But there’s usually still plenty of behind-the-scenes design that can’t be communicated this way. For that, some text descriptions entered into a wiki, along with some diagrams showing class or table relationships, is still your best bet.

People never liked technical documentation because the bulk of its maintenance is related to low-level design. If you’re documenting that this object has these ten methods and this first method is named this and takes these three arguments, each which have this type - well, that’s going to get out of date quickly. It’s a pain to maintain which means it won’t be up to date, and when it’s not up to date no one uses it and thus it becomes worthless. Unit tests free you from this painful cycle. (If you’re making a complex, publicly-used support library, reference documentation generated from the real code with something like RDoc should complement the tests.)

Cut out documentation of low-level design, though, and you’re left with much less documentation to write. I suspect most good technical designs - and by “good” I mean “free of excessive complexity” - could be summarized in a few paragraphs and a diagram or two, backed up by a larger document to cover domain-specific knowledge for the data tracked by the app.

Recently I recorded a few of my developer workshop presentations for public consumption on the Bitscribe site:

http://bitscribe.net/screencasts

Sure, screencasts are trendy, but you gotta admit they are fun. (Acknoweldgements to pyvnc2swf, which as near as I can tell is the only real option right now for recording cross-platform screencasts.)