Adam @ Bitscribe

Software is traditionally created with the same basic approach that one would take doing any other kind of engineering - say, building a bridge. The architect lays down a detailed blueprint; during construction, the design changes very little. Let’s call this up-front design.

Despite decades of building software in this fashion, no one has been able to get it to work as well as well for software as it does for bridges. Software is different because it’s soft. You can, and usually do, make massive changes to the design over the lifetime of a project. In most cases version 4.0 of a product has almost nothing in common with 1.0. It would be like if halfway through building the bridge, you decided to make it a boat instead.

This would be insane for a bridge, but - counter-intuitively - it’s actually possible with software. More than possible, in fact: it works better. This is what extreme programming proponents call evolutionary design. It’s what I like to call like to call it “just in time design” or “iterative design”. First, the bullet points.

• Take small bites
• Do the simplest possible thing that will work
• Degrade gracefully
• Release early, release often
• Don’t be afraid to change the design mid-stream
• Throw in unplanned icing when it’s easy to do so
• Drive new features based on feedback, not speculation
• Keep the momentum going

Let’s look at each in turn.

Take Small Bites

Development of discrete software components should be measured in hours or days, rather than months or years. If your design is calls for more than 200 man-hours to get to the first functional release, it probably needs to be broken into subprojects. Small bites allows all stakeholders - the product designer, the developer, and the user or user proxy (e.g. management) - to stay focused. It keeps the iteration cycle short (more on this below), which translates to agility and responsiveness. Don’t promise some enormous project then send the development team into a cave for six months to produce it. Do promise a small, relatively atomic piece of a larger project that can be turned around to them by the end of the month. It needs to be something that is functional and can stand on its own, so that it can be shown to the user or user proxy, to get their feedback before proceeding on the next iteration.

If a developer is working on the same component for more than a few weeks and still isn’t done, something is wrong. The developer and product designer should get together and rework the design to break it into smaller pieces. If the developer and designer are the same, it’s up to them to whittle it down to size.

Simplest Possible Design

When deciding how to implement a new component or feature, choose the simplest thing you can think of that has any chance of fulfilling the project’s goals.

This goes for both the design and the code itself. You may find it counterintuitive in both areas. Most people who have worked with software creation are accustomed to plotting out grand designs which try to incorporate every possible feature imaginable. The implementation often goes even further, building complex code infrastructures for handling not only present needs but future ones.

Trying to anticipate future needs is a particularly nasty trap. Never add functionality early. Instead, just repeat this handy phrase to yourself: “You aren’t going to need it.” Agile development means that features can be added exactly when they are needed. Trying to add them before then is a waste of everyone’s time, and worse, clutters up the code.

You may be surprised at what the “simplest possible design that could work” actually is. For example, does that e-commerce site you are building really need to be able to put products into multiple categories? Creating a many-to-many relationship between products and categories is much more complicated than just having one product per category. The user may swear up and down that they need that feature, but do they really? Get a list of products that need to go in multiple categories. Chances are good that it won’t be nearly as vital as you or they might think.

Or, to quote an old maxim: Keep it simple, stupid! (KISS)

Degrade Gracefully

Doing the simplest thing means leaving out as many features as possible. But, someone will inevitably argue, “we need those features so that we can do [insert task here]!” The solution is to degrade gracefully. This means that coming up with a simple stand-in method to handle cases that are not fully supported in the current version.

For example, imagine you’re building a shipping calculator for an e-commerce site. The calculator does some fancy calculations to find the distance between the zipcode of the shipper and the zipcode of the receiver, combined with the weight of the products in the order. This covers 95% of the merchant’s orders, which are mostly from domestic.

They do, however, have an occasional international order. In designing this component, the product designer would be right to say “We aren’t supporting international calculation in the first version.” But the merchant will protest: “But we need international shipping! 5% of our orders depend on it!

A graceful degradation should be chosen that will serve as a temporary solution until a future version. Manual fallback is always a good way to degrade; for example, all international orders could be forwarded to one of the merchant’s staff for manual processing. In this case, a quick-and-dirty estimation algorithm is probably in order. This might look like:

if @order.is_international
# We don't support true international shipping calculation yet, so this is a
# quick-n'-dirty stand-in.  Joe @ the merchant said that the prices this
# algorithm produces should be sufficient to cover their true shipping cost.
shipping_cost = 10 * @order.weight
else
shipping_cost = fancy_domestic_shipping_calc(@order)
end

This is a graceful degradation because it allows the order to pass through the system. The comment clearly indicates that the method is temporary, and where the code should be changed when the time comes to make a full-featured international calculator. By contrast, a non-graceful degradation would be something like printing an error message and blocking the order from passing through whenever a user tries to place an international order.

One thing you might be surprised by is how often graceful degradations stay in much longer than the label “temporary” implies. But this isn’t a bad thing in the slightest. If you introduced a simple algorithm that was easy to implement and served the user’s need, then everyone wins.

Release early, release often

If we take small bites and do the simplest thing that could possibly work, the iteration cycle is inexpensive. As a result, frequent releases are easy. By iteration cycle, I mean the complete process of:

1. Design
2. Implement
3. QA
4. Deploy
5. Collect user feedback

In conventional software methodologies, this whole process takes years. Web application development has reduced this time quite a bit - deployment is a matter of updating a server, not getting boxed software onto the shelf. Just-in-time design takes it even further. At Bitscribe, for example, we interact directly with our users as clients. So it’s not uncommon to have the client ask for a new component on Monday; the project manager delivers a design Tuesday; the developer implements it by Thursday; it is QA’d and deployed by Friday morning and the client has sent feedback by the following Monday.

An important part of this point is frequent integration. Modern version control tools make code integration a snap as long as merges are frequent. You may be tempted not to commit because “it’s not done yet,” but this only makes the eventual merge more painful. In particular if someone else is working on the same area of the code, an early merge will produce conflicts and let you know you should be coordinating with them. A lot of time may be saved by avoiding duplicate (or conflicting!) work.

This is not to say that you should commit broken code. The first part of this is to make sure that your code itself is done in small bites. You should be able to commit once a day, even if you’re working on a larger project that won’t be functional for some time yet. The idea is that you can commit a working piece of that larger project, to start getting the footprint of your component into the codebase - even if it’s not called by anything.

It’s also advisable to commit to the development branch of your version control repository, which will make you more willing to check in things that aren’t thoroughly tested. This implies a process with one or more staging servers to test new code and prepare it for production in deployment. But this process, too, should be kept to a short iteration cycle - a few weeks, perhaps a month. Too long between creating the code and seeing it in production loses much of the benefit of iterative development. It also keeps the project vital and the team excited, since they soon get to see their latest work in action.

Don’t Be Afraid To Change The Design

The design specification should serve as a loose guideline for the project. It should define the goals of the project and then a proposed implementation. But once you, the developer, dive into the code, you’ll often find that the implementation described in the design is not the best possible approach. In that case, don’t hesitate to change the design to produce the best possible outcome. The design is a starting point, and if it was well thought out, it should carry you through most of the implementation. But there’s always at least a little bit of deviation, and often quite a lot.

For example, let’s say you’ve got a task which specifies that users can create, edit, and delete inventory items. In working on it, however, you realize that deleting inventory items will not work, because they are linked via foreign keys to inventory events, which must be stored forever for archival purposes. What should you do? If you treat the design as an immutable mandate from on high, then you may find yourself spending a huge quantity of time trying to create some elaborate solution like relinking all the event records to a shadow inventory object. On the other hand, treating the design as fluid makes it possible to step back and say: what was the goal of being able to delete inventory items? There’s a good chance it was included in the design spec just because deletion is a part of the standard CRUD approach. If so, you can just drop this feature from the design. The designer probably thought it would be easy (usually it is), so they threw it in there. But there may be a specific user need for it, and therefore no reason to spend a lot of time on it. (Don’t forget that code costs more than just the time spent creating it: it also needs to be maintained forever.)

Even if the user does need deletion, remember to do the Simplest Possible Thing That Could Work. For example, maybe “deleting” inventory items shouldn’t delete them at all, but instead their quantity to zero. Then the page which displays inventory only shows items which have a non-zero quantity. This can be wrapped in a method or database view named something like “active_inventory”.

In some cases you will get into a project and realize that the entire design is iffy. In that case you and the product designer should go back to the drawing board. Don’t just try to muscle through. You waste everyone’s time by doing that.

A good way to judge this sort of thing is simply by gut feeling. Are you working on a feature which seems to be dragging on and on, that - no matter what you do - just keeps sucking? That’s a sign that the feature needs to be changed or dropped altogether. Good features are ones that keep you excited while you’re working on them. You find yourself thinking things like “Man, this feature is sooo easy to implement, and the users are going to LOVE it!” This sort of low-hanging fruit is exactly where software development resources should be spent.

Throw In Unplanned Icing

So, we’re willing to drop not-so-useful, hard-to-implement features specified in the design. The mirror of this is unplanned icing: useful, easy-to-implement features that aren’t in the design.

Usually, icing comes in the form of small usability items: an extra navigation link between two pages with related data, a smart default on a dropdown, or a hotkey to invoke a frequently-used function. Whatever it is, it will be driven by your intuition. You’re using a page and find that it’s annoying to test because you have to click through a bunch of menus to get to a certain function. Chances are, this will be annoying for the end users too. A five-minute feature addition could vastly improve usability, so throw it in there.

You do have to be a little careful with this. What may seem like a five minute addition could turn into thirty minutes, or an hour, or a day. In that case you probably want to coordinate with the project manager to make sure there’s sufficient time available, or just bounce the idea off someone else to make sure they think it’s cool enough to be worth spending a day on.

Drive New Features From Feedback, Not Speculation

We’ve already covered not designing for the future (remember, You’re Not Going To Need It). Let’s extend this a little further. What should decide which features will be included in the design? There is pretty much only one answer: user feedback. Talking and observing the software’s end users will make it crystal clear which features would be highly beneficial right now. There are always 100 times more features hanging around on the wishlist than there is time to do. Using this method of real-world analysis is the only way to effectively prioritize such a massive list. Over time, steady creation of the most beneficial features - as judged on an ongoing basis - produces astonishingly useful software.

This is also why the initial implementation should be absolutely rudimentary. Following this approach, the entire lifecycle of a software product should be something like:

1. Talk to users/user proxy/client and get a vague sense of initial needs
2. Come up with a rudimentary design that can be implemented in a very short timeframe
3. Do the implementation and show it to users as soon as possible
4. Create a design for the next version based on features driven by their feedback
5. Goto #3

It’s important to note that “user feedback” is not the same thing as “listen to the user describe the features they want.” You should be using your own judgement about what changes should be made based on watching users use the software. What they ask for plays into this too, but actions speak louder than words. You’re the software expert, so you need to combine your own experience and expertise with careful observation of the software’s use in real-world settings. (Usability testing via videotape or webcam is a great technique for gathering this type of user feedback.)

Just-in-time design, iterative development, evolutionary design, agile development - call it what you like. Small steps with copious feedback produces applications that are closely synchronized with reality and therefore massively more beneficial to its users compared to long-iteration, carefully planned software.

Keep the Momentum Going

If you start following all of the rules above, you’ll find that development is divided into short, bite-sized chunks which go through the complete design-implement-release-feedback cycle very quickly. You blaze through implementation of good features (because they are EASY!) and redesign or drop slow, grinding features before getting too far on implementing them. You’re on a roll: now all you’ve got to do is keep it going.

Once you get used to slamming through major feature additions and changes at a breakneck pace, it should be really obvious when you hit something that slows you down. If you find the time between your commits getting longer…time spent on a single task or set of tasks to be more than a few days…staring at the same piece of code day in, day out - you know something’s wrong. Take the design back to the drawing board or do whatever it takes to get back to your usual fast turnaround.

Keeping your momentum up makes users, management, and/or clients happy, your teammates enjoy working with you, and your job satisfying and fun. It may take some getting used to, but believe me - once you’re in the habit, you’ll love it.

This entry was adapted from a section I wrote for the Bitscribe developer’s handbook.

This entry was posted on Thursday, May 25th, 2006 at 4:48 am and is filed under Design. You can follow any responses to this entry through the RSS 2.0 feed. Responses are currently closed, but you can trackback from your own site.