Reflections on software

Best practices are constantly evolving. Sometimes so much so that we're hearing the opposite today of what we heard yesterday.

But there are some fundamentals that tend to stay the same though. One that most of us can agree one is that bugs can be expensive. More specifically, the later we find bugs, the more costly they are to fix. On a whiteboard, the cost of fixing a bug is just about nothing. It can be as simple as replacing one diagram with another. But in production, the costs could be catastrophic, including loss of current or potential customers, direct loss in financial applications, or human life could even be at stake.

On one side of the spectrum, we have the school of thought that tries to solve as many problems initially as possible. After all, if it's cheaper to fix bugs earlier in the process, let's just spend more time earlier in the process. In practice, this doesn't work so well for a number of reasons. The biggest one is probably just the fact that requirements change during the process itself. There's not really anything we can do in this case. The other pitfall here is that most tough problems in the programming space are relatively new. We don't have much experience with these problems, so it's difficult to account for or even predict the tough spots. Usually we have much more insight into problems after we've tried to solve it the wrong way a few times.

I think what typically ends up happening with this approach is that some areas get over designed, and others don't get the attention they deserve. This can lead to getting the abstractions wrong, and leaky abstractions can make bugs difficult to prevent. Not knowing what invariants need to hold for a system can be a big source of errors.

The extreme/agile approach looks at it from a different perspective. Instead of trying to imagine all the scenarios and details up front, let's just do what we can to discover them early in the process. To use a general oversimplification, it boils down to ignoring complexity in some areas in order to create a working prototype faster. The trick then is knowing which parts of the problem to ignore and which to focus on. Trying to tackle the hard parts is usually a solid strategy, except sometimes the hard parts aren't what we think they are.

Agile development also claims to engender quick changes. If the abstractions still fit, and we have a testing suite underneath us, then yes, the changes can be really fast. But if the abstractions fit, then any system is fast to change. When a requirement change breaks the current model though, or forces an interface change, then I would argue that making changes can be even slower, because they propagate out to more code. We not only have to refactor our logic, but the logic in all relevant tests as well. Some would argue that means the tests aren't written well or aren't testing at the right level. But writing tests at the right level is hard. It can take some time before we can learn how to write effective tests, just like it takes time before we can learn how to write effective code.

But the biggest concern I have with agile development is that it can be used as an excuse for poor judgement. Being lazy is not agile, it's being lazy. Always taking the easy way out usually catches up to us, and then it can be painful to dig out of that hole.

Here are my takeaways:

1. If we don't understand the problem, we can't process our way out of it. In other words, if we don't know what the issues really are, no amount of process can help us.
2. We learn best from our own mistakes. This informs our future decisions much more so than any process could.
3. Context matters most.

Generally speaking, when we think of adding value, we usually think of what we can add to make improvements. But another way we can make improvements is to remove what's not essential. By trimming out what's not needed, focus is shifted back towards what's important. In some cases, these subtractions can actually add up to a whole lot.

This perspective is usually directed towards design. And for good reason. Simpler designs are friendlier, easier to use, and generally yield better outcomes. But we don't always share this perspective when thinking about software. Frameworks, apis, even programming languages themselves can all benefit from simpler perspectives. Strangely enough though, complexity here is often touted as an asset instead of a fault. In fact, the implicit reasoning is usually that things need to be complicated for them to be worthwhile.

Sometimes we even inflate simpler problems into harder ones. This serves as an excuse to implement more complex solutions. Unfortunately we can do this without even realizing it. And the more abstract the original problem is, the more room there is to introduce complexities. In some cases, it might be easier to err on the side of solving none of the abstract problems instead of trying to solve them all.

So if we find ourselves several layers removed from the original problem, it might be time to take a step back and evaluate how we got here. It might be the right path, or it might not. But if it's the wrong path, then we should turn around.

We try to keep things simple. After all, nobody starts a project thinking, let's make this one complicated for a change. Yet somehow, things like to end up that way.

I like to think of this as a shock, but it's usually not that surprising, all things considered.

Perhaps the most obvious reason is that programmers knowingly introduce this complexity themselves. This complexity is justified by rationalizing that it will anticipate future situations, and account for them elegantly. The goal here is to prevent future maintenance headaches, but ironically it can add complexity if the wrong abstractions are made.

The opposite can also be just as true. As code design debt accumulates, not acting early enough once a problem is recognized can also lead to increased complexity.

Programmers can also have drastically different opinions on subjects. If we take a look around the software world, we'll see that there are many operating systems, langauges, frameworks, and a myriad of all kinds of tools. Each one of these represents a particular area where a programmer identified a gap, and then proceeded to make an attempt at filling it in. This naturally leads to overlap among similar software. This overlap itself isn't bad, in fact it's probably a little bit healthy. Each competitor can focus on what it feels is important, and users can choose accordingly. But merely having more anything can contribute to its own kind of complexity to deal with. After all, it's easier to make a choice given fewer options.

But along with the lack of consensus on what's important, programmers also don't think the same things are unimportant. Some are willing to let documentation drift out of sync. Some aren't interested in repeatable builds. How about automated tests? Commented code? Consistent style? Design patterns? Stable back ends? Intuitive front ends? Practically everyone will agree that it's important to have many of these items, but not everyone will agree on which one is ok to let slip. These differences can help contribute to software complexity as programmers fight against each other.

Programmer 1: I never knew it would be so hard to push a car downhill.
Programmer 2: What do you mean, downhill!?

There's also the idea that things aren't worthwhile unless they are complicated. When things are simple and easily understandable, it's easy to suggest avenues of improvement. But if too many distractions are entertained, the codebase can start developing warts that can be difficult to deal with.

It is easy to make things. It is hard to make things simple.

Constantly learning is important. Lots of disciplines today change at an extremely fast pace. Software is definitely one of those fields. As we learn new things, we try to integrate them into our existing mental frameworks. We think of different areas where we can apply this new knowledge, or if it can be used to simplify any existing solutions. I for one am constantly trying out new frameworks or languages by thinking of apps I can write with them :)

There is a caveat here though, particularly with newer programmers. Right after learning a new pattern, we have the propensity to use it more than we should. We force it to fit our problems, usually by throwing more patterns in the mix. The code ends up being far too complex for what it's supposed to be doing, and it's much harder to follow what's actually going on. In short, we end up with more complexity and less flexibility. That sounds like what we were trying to avoid in the first place.

Programmers also tend to get emotional and invested in their solutions. Admitting that we took the wrong path is hard to do. Software is the kind of thing that doesn't really have just one answer; we can solve our problems with just about any language or technique. Some are obviously better for certain problems than others, but just about any of them can be *made* to work. This brings me to a quote:

xml is like violence. If a little doesn't work, just use more of it.

I've got more quotes about xml, but I'll save those for another day :)

We all have our own opinions on what kinds of software or tools are useful, and many of us have very strong opinions on these matters. My opinions here tend to change on a daily basis :) But even though we don't like to admit it to ourselves, virtually every approach has its own tradeoffs. In fact, I'd go so far as to say that most of programming is just a series of tradeoffs.

The important thing is to use the best tools for the job. Having lots of opinions is healthy, and it only really becomes dangerous when we fail to consider other options. Bad options are valuable to us too, because they help us focus on what really *is* important.

Perhaps what's most striking is the fact that our problem solving methods in the software world are sometimes backwards. We have a list of solutions in mind, and we try to see which one fits the problem. From one perspective, this is completely natural. How are we supposed to think of solutions that we don't know of yet? But on the other hand, none of our pre-canned solutions may be a good fit at all. In this case, some research into the problem space may help clear the muddy waters and reveal better alternatives. But how do we know if this is necessary, and we're just not wasting our time? Well, I like to follow the mantra that if it feels like there must be an easier way, then there probably is. In any case, taking a step back and looking around again can be a healthy thing. After all, times change, perhaps faster now than ever before. Yesterday's pitfalls can become today's best practices.

When all you have is a hammer, everything looks like a nail.

May you live in interesting times

Never before has access to information been so easy, and so readily available. And we're starting to accumulate more and more at an incredible rate. What's more is that we are are able to summon information virtually instantaneously. But ironically, too much information can also have the effect of preventing any useful information from getting out. In addition to having a wealth of information readily available, we're also forced to filter out the noise to find the gem in a haystack, if you will.

But if we find ourselves constantly filtering the same kind of information, computers can help us here. After all, they're good at repeating tasks over and over again. And here is where programmers have a significant advantage.

Programmers can improve their rate of learning.

In particular, programmers are able to improve their workflows. They're trained to recognize repetitive patterns, and can make tweaks here and there to help things along. But when you can take baby steps quickly, you start to move fast.

An example that comes to mind is the tool for os x: quicksilver. If you're using os x and you haven't checked it out, you really should. If you're running gnome, gnome do provides similar functionality. These tools let you express the action you're trying to perform more naturally, almost in the form of a sentence. But command line users have enjoyed this richness for a while now. This is an example where the programmers are ahead of the curve, but it's good to see that there are efforts to help out the "less fortunate" :P

Throughout your life advance daily, becoming more skillful than yesterday, more skillful than today. This is never-ending.

We can see software development as a duality with beauty on one side, and practicality on the other. Beauty is coming up with a masterpiece of a solution. Practicality is solving our problems with methods that are known to work. These don't have to be mutually exclusive, but they usually feel that way.

The advantage of the practical solution is that we know it will work. We've usually employed it in the past, and we're confident with it. Examples of brute force methods come to mind, although they don't necessarily have to be. The practical approach is particularly appealing to business, which almost without fail tries to avoid the worst possible scenario. Regardless though, we know we will end up with a working solution in a fixed amount of time, and not just with only scribbles on a whiteboard.

The problem with this is that things can get difficult to work with after a while. Like trying to find something on a messy desk, it's always easier to quickly go through the mess. But if we repeat this several times, then it's better to organize the desk appropriately first.

Finding an elegant solution can mean modeling our needs as a specific instance of a more general concept. We can also try to fit our needs into a pattern by thinking about them from a different perspective. Generally though, I think elegant solutions share the fact that they make our problems look simple.

Looking simple is key. When things are simple, we are able to have many more ideas freely dance about in our heads. Merely keeping complex ideas in our heads can be difficult, let alone finding ways to extend them. And at this rate, we can forget about debugging.

But beautiful solutions are more than just simple. They are a set of neat abstractions, naturally joined together. Each addition compounds on the previous, magnifying the strengths of the whole with a new dimension of power. This continues until the problem itself looks feeble in comparison, like it was created as a reaction to the solution.

Unfortunately, it's difficult to create these problem slayers. Our problems usually hide in dark corners, only revealing themselves at inopportune moments. It's only then that we realize that our methods are inadequate, and sometimes it can be hard to adapt them. It's at this point that we often see the obviousness of our shortcomings. But of course, it's always easier to predict the past than the future.

In the end then, we need a healthy mix of both beauty and practicality. We need to cultivate our magnificent visions, while listening to the sobering voice of reality. As Alan Kay said:

Just play it grand.

Most programmers follow a progression as their skills improve.

Initially, we have a just make it work mentality. In this phase, there is no structure or attempt at creating any abstractions. If functionality needs to be duplicated in a slightly different fashion, the lines of code get duplicated. We happily trudge along in this phase until we are met with a project of significant complexity. Here we start to break down, and quickly learn that it becomes difficult to maintain.

In the next phase, which I like to call "abstraction envy", we start to learn the different ways we can architect our applications. We begin to learn the different designs, structures, and patterns that we can use to ease the burden of maintaining applications. As we learn more and more patterns however, we try to apply our newly found knowledge as widely as possible. Given that this is still a learning process, and much of programming is learned by doing, we often pick the wrong tool for the job. To use the popular metaphor, we use a hammer, but we're not hitting a nail. We slowly start to learn that their are some downsides to these patterns as well, and just because we understand them doesn't mean we should use them. If a little is good, that doesn't mean that a lot is better.

This next phase I'm about to describe is the middle ground between the first two. This ideal phase is essentially a zen state of programming. Abstractions are only used when they are necessary. And the abstractions introduced are not just for code reuse, but they create simple, yet powerful ways of thinking about our problems. These types of abstractions are more useful because they more closely match the conceptual, or real problems. You'll know when you see one of these because it'll seem like the problem was created for the abstraction instead of the other way around. Besides creating perfect abstractions for the parts of the application that need them, the flip side is just as important, if not more so. Abstractions are not created for the parts of the application that don't need them. Simple things are left simple, and complicated things are possible.

I think most programmers are in the middle state. If you ask a programmer his advice on a particular problem, you'll probably get an answer, and you'll probably get a lot more information about other problems that get created along the way. Sometimes these problems themselves require complicated solutions, and then these problems cascade.

That brings us to the heart of the problem really. Simple things should be easy to change. If they're not, something is wrong. But none of us writes code with the idea that we're making certain things harder. We always believe that we're improving things. We see some boilerplate, and we think to ourselves, gee, wouldn't it be great if we didn't have to do that all the time? Let's find a way to eliminate it. We move along happily, proud of how much code we've reduced. The problem lies in the future, when we're thrown a curve ball. It doesn't fit in our strike zone, but we've got to hit a home run anyway. We look at the code, and we realize that it would have been a whole lot easier to do if we didn't have to deal with those abstractions we added in earlier. We'll have to modify them significantly to make the new problem fit. In other words, the abstraction leaks, because we now have to understand the implementation.

And now we're faced with a tough decision. Do we make these modifications - modifications is a good word for this, it's usually a hack - or do we scrap them and go for a simpler approach. The hacks are usually easier and we feel more confident that they'll work, but they increase the entropy of the system which will make it harder for us to maintain. A bigger cleanup will take more time and is riskier, but is probably a good long term investment in some cases. Unit tests will help in this case, but like good abstractions, I find that it's just as hard to write or find tests at the right abstraction level to pull this off.

So how can we avoid getting ourselves into a mess like this in the future? I think the real answer is that we have to get ourselves tangled into a few webs of anarchy before we can learn how to avoid them. But in an attempt to answer the question, instead of erring on the side of introducing abstractions, we can err on the side of leaving them out. Simpler code is usually easier to modify, although that's not true for large pieces of spaghetti where a few well placed abstractions trim out a lot of the complexity. But my personal opinion is that it's usually easier - and more fun - to add in additional layers of abstractions rather than break free from existing ones. It also feels safer.

But perhaps what should guide us the most in these ambiguous cases is the principle of least astonishment. If the presence of a particular abstraction is surprising, then it probably shouldn't be in there. Another good question to ask ourselves is if the new abstraction actually makes things simpler. If the code is easier to understand with the abstraction, awesome. If it's a puzzle to figure out what is going on, and it's just there to save on lines of code, change it.

It's easy to make things. It's hard to make things simple

It's interesting to hear so many different opinions on the command line. Many see it as archaic. Others are scared of it. And then there's a few who prefer it to all other interfaces. But regardless of how you feel about it, there's no denying that it's here to stay, at least for the near future.

I've mentioned earlier that interacting with a computer can be likened to communicating with it. We usually "speak" to it in the physical sense through a keyboard or mouse. On the virtual or software side of things, the most popular interfaces are either through the command line, or a gui (wimp interface).

Command line interfaces allow for more direct communication with the computer. You type words that you want the computer to execute, and it returns the response. Going through a gui is like talking to an interpreter first, and then having the interpreter relay the information. This can be more useful if the interpreter can figure out what you mean and make a more informed request to the computer. But for informed users, interpreters just get in the way.

Where I think the command line really shines is its flexibility. At a moment's notice, you have access to virtually anything, all through a single interface. You have access to a large, powerful set of tools that can be widely used with one another. For example, what if I wanted to count the number of files in a directory that had an odd number of lines in them? That were edited in the last week. And have those sent out in an email. Biweekly. Granted, that's a contrived example off the top of my head, but one that is virtually effortless to accomplish from the command line, yet difficult with a gui that wasn't designed specifically for that.

Another useful feature of the command line is its inherent repeatability. Once a command has been run, it can be recalled, executed as is, or executed with slight variations, all with little effort. This is true for sequences of commands as well. And, if the sequences themselves start repeating in sequence, then they can be moved to a shellscript or function and run with a single command. In this way, the command line allows for the user to create new language that is better suited for the problem, in a bottom up approach.

Guis have their place too though. The knowledge required is usually much lower to start being effective with an application. And guis are a better fit to solve visual problems. It's much easier to work with a wysiwyg type app when you're producing a new design, rather than having to perform transforms with commands and then redisplaying.

But I think the audience is what makes the biggest difference. Programmers, and "expert users" have a tendency to prefer command line tools. Novice users are usually afraid of having something go wrong, and find comfort with gui applications. And for the die hard gui guys, all you have to do is tell them:

Smith and Wesson was the original point and click interface

Programming is really just another form of communication. We type in some code, tell the computer that we've got some stuff for it to read, and then it gives us back a response when we run it. The write, compile, run feedback loop is like having a conversation with the computer.

Generalizing a little bit, using a computer itself is a form of communication. We put some input into the computer, whether we type something in, or use a mouse or some other device. Then the computer responds back to us. It can also work the other way too. The computer can prompt us for action, usually after a particular event occurs. For example, we can have a msgbox alert us of new mail as it arrives.

Thinking about it from a conversational perspective, our interactions with the computer become more effective if we can better articulate what we are trying to do. If we can phrase a question, or a particular action in a more effective manner, then it's more likely that we'll get a good response back from our computers. Usually, the whole trick comes down to saying the right thing.

And as programmers, the choice of programming language is really just that. It's how we choose to describe our application to the computer. Seen from this light, arguments have been made that if you're able to express yourself in fewer words, then you're using a better language for the job. It also follows the language parallel in that certain languages have many more words for a particular concept, each with a slightly different meaning. These languages are adept at expressing the subtle variations of this concept. And if a language doesn't have a word for a concept, then chances are it doesn't know how to express it effectively either.

Programmers generally get a bad rap about being poor communicators. But, if programming is just a form of communication, can this be true? I'd argue no. The best programmers turn out to be effective communicators too. They really are able to express their points in a clear, concise way. After all, they're used to writing clear, concise code.

When discussing whether quality is more important than quantity in programming circles, quality will often be cited as the clear winner. The argument is that focusing on quantity only ends up hurting us in the long run. Sacrificing quality usually means taking so called "shortcuts", which can lead to headaches in the future. When the shortcuts turn into dead ends, we end up having to take detours to get around them.

But does focusing completely on quality necessarily improve it? We try to justify spending more time improving quality by saying that once it's done the right way, the problem is less likely to resurface in other ways. In other words, we're spending more time on it now so that we don't have to in the future. But what does it actually mean to concentrate on the quality of the application? Just because I try to anticipate future uses of the code, or remove all duplication, or try to document what I'm doing doesn't necessarily mean the code is of higher quality. If there are known bugs and I eliminate those, I haven't necessarily raised the bar quality-wise. I could have introduced other defects as side effects, or some security or performance issues. I can spend more time writing tests, but again, I haven't improved the quality of the code here.

I'm not going to make an attempt to define software quality myself. I only wanted to point out that it's not completely obvious, and refactoring has its own risks. Instead, I'll summarize an interesting story I came across here.

In a ceramics class, half of the students were graded completely on quantity of work produced. The other half was graded completely on quality. You'd think that the quantity students would produce tons of sub par work, while the quality students would produce one amazing work. I did anyway.

I was wrong. The quantity group in fact ended up producing works of higher quality. They were able to learn from their own mistakes. The quality group, while coming up with sound theoretical works, failed to deliver.

This sounds very similar to programming. And unless you're this guy, it usually takes a few iterations to get something right.

We tend to learn more effectively through our own mistakes. Just like parents will let their children make their own mistakes, programmers learn to avoid pitfalls by first falling into them. The better programmers are the ones that write more code.