The Success of Failures

December 31, 2011
Once a group of ten blind masseuses were travelling together in the mountains, and when they began to pass along the top of the precipice, they all became very cautious, their legs shook, and they were in general struck with terror. Just then the leading man stumbled and fell off the cliff. Those that were left all wailed, "Ahh, ahh! How piteous!" But the masseuse who had fallen yelled up from below, "Don't be afraid. Although I fell, it was nothing. I am now rather at ease. Before falling I kept thinking 'What will I do when I fall?' and there was no end to my anxiety. But now I've settled down. If the rest of you want to be at ease, fall quickly!"

To err is human. But we often think of mistakes as necessary evils, actions or situations that could have been avoided if we had the foresight. After all, some mistakes can be painful. Yet, I would argue that we learn best from our own mistakes. Even if it's not a conscious thought in our mind, when faced with a similar situation or problem, our intuition can help us navigate around repeating the same mistake twice. If I touch a hot stove once, I'm probably not going to touch it again.

For easier problems, the cause and effect between the mistake and the outcome is readily apparent. But as things get more complicated, it's not always easy to see what the actual mistake was that generated the failure. From a software development perspective, the actual underlying cause can elude us, and everybody can be left drawing their own conclusions as to what happened. We need to be careful here though, because the wrong lesson can guide us down the wrong path in the future. Once we are burnt by a hot stove, we'll never touch a hot stove again. But if we learn the wrong lesson, we may never touch a cold one either.

One interesting idea is that the problem space itself can dictate the strategy used to solve it. When all variables are known, we simply use the answer for our given permutation. However, some problems don't have an easy "if this, do that" answer. For these problems, we can set up fail-safe experiments, where each one is an attempt at a solution from a different angle, but their failures aren't catastrophic. Recovering from the failures is the key here. In fact, many failures initially can lead to a better outcome in the end, because they can each inform the ultimate solution based on what we learned from their failures.

From a business perspective, this can be a hard sell though. How can you justify allocating resources on what you know will mostly end up being a failure? Isn't that just a waste? What we need to admit first is that we may not know enough about the particular problem to be in a position where we can recommend a single solution that has a good chance of success. And the best way to learn more may be to attempt to solve it in multiple ways, many of which will fail. Naturally, nobody wants to hear this kind of news. The immediate reaction could be, "well, let me try to find somebody that knows more about this." But for problems that are relatively new, experts can be hard to come by.

Some will also try to rely on a process to get through the problem. And for known problems, it is a fine approach to rely on best practice. By definition though, best practice is past practice, so we can't expect to have best practice for all situations, especially for new problems that we don't fully understand.

Accepting that failures occur is an important step. Instead of focusing on preventing them completely, we can instead create environments that are more tolerant of our failures. And we shouldn't simply tolerate mistakes, but accept them as an integral part of the process, and how we continue to improve ourselves.

At the time when there was a council concerning the promotion of a certain man, the council members were at the point of deciding that promotion was useless because of the fact that the man had previously been involved in a drunken brawl. But someone said, “If we were to cast aside every man who had made a mistake once, useful men could probably not be come by. A man who makes a mistake once will be considerably more prudent and useful because of his repentance. I feel that he should be promoted.” Someone else then asked, “Will you guarantee him?” The man replied, “Of course I will.” The others asked, “By what will you guarantee him?” And he replied, “I can guarentee him by the fact that he is a man who has erred once. A man who has never once erred is dangerous.” This said, the man was promoted.

Being functional

January 10, 2011

Many have written about why functional programming matters. There's a great pdf on why functional programming matters by John Hughes that I highly recommend reading. It talks about the many benefits of code written in a functional style, that it's incredibly more composable, testable, and in short, higher in quality.

But functional programming is also important for another reason. It's more awesome.


fac n = foldl (*) 1 [1..n]


Computing factorials is the classic recursive example.


fibs = 0 : 1 : zipWith (+) fibs (tail fibs)


This corecursive example generates a lazy list of the fibonacci numbers in linear time. The cool part here is the self-referential data structure. I like to call this the reverse ouroboros definition: the tail eating the head.


qsort [] = [] qsort (p:xs) = qsort lesser ++ [p] ++ qsort greater where lesser = filter (< p) xs greater = filter (>= p) xs

Haskell (copied from literate programs wiki)

This is beautiful.

Find all numbers which have only 2, 3, & 5 as non-trivial factors.

from itertools import tee, chain, islice, groupby from heapq import merge def hamming_numbers(): # Generate "5-smooth" numbers, also called "Hamming numbers" # or "Regular numbers". # See: # Finds solutions to 2**i * 3**j * 5**k for some integers i, j, k. def deferred_output(): 'Works like a forward reference to the "output" variable' for i in output: yield i result, p2, p3, p5 = tee(deferred_output(), 4) # split streams m2 = (2*x for x in p2) # multiples of 2 m3 = (3*x for x in p3) # multiples of 3 m5 = (5*x for x in p5) # multiples of 5 merged = merge(m2, m3, m5) combined = chain([1], merged) # prepend start output = (k for k, v in groupby(combined)) # eliminate dupes return result

Python (copied from ActiveState)

This cyclical iteration technique in python is the same kind of idea as the haskell fibonacci example above. The output streams are fed back in to generate the final result.

List flattening

(defun flatten (x) (labels ((rec (x acc) (cond ((null x) acc) ((atom x) (cons x acc)) (t (rec (car x) (rec (cdr x) acc)))))) (rec x nil)))

Common lisp

An example of a doubly recursive utility.

Function intersection

(defun fint (fn &rest fns) (if (null fns) fn (let ((chain (apply #'fint fns))) #'(lambda (x) (and (funcall fn x) (funcall chain x))))))

Common lisp (copied from onlisp)

Example of a function builder, with a recursive definition. The result here is and'ing the functions together. This allows us to say things like:

(find-if (fint #'signed #'sealed #'delivered) docs)

Recursive function generators

(defun lrec (rec &optional base) (labels ((self (lst) (if (null lst) (if (functionp base) (funcall base) base) (funcall rec (car lst) #'(lambda () (self (cdr lst))))))) #'self)) ; copy-list (lrec #'(lambda (x f) (cons x (funcall f)))) ; remove-duplicates (lrec #'(lambda (x f) (adjoin x (funcall f)))) ; find-if, for some function fn (lrec #'(lambda (x f) (if (fn x) x (funcall f)))) ; some, for some function fn (lrec #'(lambda (x f) (or (fn x) (funcall f))))

Common lisp (copied from onlisp)

If we find ourselves constantly building recursive functions ourselves to traverse through a list, why not abstract out that pattern? Here we see a technique for doing just that. We just need to define a function whose first argument represents the first element of the list, and the second is a function to call to continue the recursion. We can also define traversers on subtrees, which the reference link explores.
Note that we can make this even more concise, but we need lisp macros for that. But that's a topic for another discussion :)

Find the maximum profit buying and selling a stock one day

(defn max-profit [prices] (reduce max (map - prices (reductions min prices))))


I just had to mention this one because of its brevity. It's almost like the problem was created to fit the solution.

20 questions

(defvar *nodes* (make-hash-table)) (defun defnode (name conts &optional yes no) (setf (gethash name *nodes*) (if yes #'(lambda () (format t "~A~%>> " conts) (case (read) (yes (funcall (gethash yes *nodes*))) (t (funcall (gethash no *nodes*))))) #'(lambda () conts))))

Common lisp (copied from onlisp)

20 questions in a dozen lines. Not too shabby.
Here we see a technique for representing structure through closures themselves. Traditionally this is modelled using a tree to represent the yes/no decisions. In this case we're able to use the functions themselves to represent the decision trees, with the state captured through closures.
If the network will not change at run time, an optimization can be made to find the references to the yes/no functions at compile time instead of looking them up through a hash table. The reference link above explores this strategy.

Best practices are constantly evolving. Sometimes so much so that we're hearing the opposite today of what we heard yesterday.

But there are some fundamentals that tend to stay the same though. One that most of us can agree one is that bugs can be expensive. More specifically, the later we find bugs, the more costly they are to fix. On a whiteboard, the cost of fixing a bug is just about nothing. It can be as simple as replacing one diagram with another. But in production, the costs could be catastrophic, including loss of current or potential customers, direct loss in financial applications, or human life could even be at stake.

On one side of the spectrum, we have the school of thought that tries to solve as many problems initially as possible. After all, if it's cheaper to fix bugs earlier in the process, let's just spend more time earlier in the process. In practice, this doesn't work so well for a number of reasons. The biggest one is probably just the fact that requirements change during the process itself. There's not really anything we can do in this case. The other pitfall here is that most tough problems in the programming space are relatively new. We don't have much experience with these problems, so it's difficult to account for or even predict the tough spots. Usually we have much more insight into problems after we've tried to solve it the wrong way a few times.

I think what typically ends up happening with this approach is that some areas get over designed, and others don't get the attention they deserve. This can lead to getting the abstractions wrong, and leaky abstractions can make bugs difficult to prevent. Not knowing what invariants need to hold for a system can be a big source of errors.

The extreme/agile approach looks at it from a different perspective. Instead of trying to imagine all the scenarios and details up front, let's just do what we can to discover them early in the process. To use a general oversimplification, it boils down to ignoring complexity in some areas in order to create a working prototype faster. The trick then is knowing which parts of the problem to ignore and which to focus on. Trying to tackle the hard parts is usually a solid strategy, except sometimes the hard parts aren't what we think they are.

Agile development also claims to engender quick changes. If the abstractions still fit, and we have a testing suite underneath us, then yes, the changes can be really fast. But if the abstractions fit, then any system is fast to change. When a requirement change breaks the current model though, or forces an interface change, then I would argue that making changes can be even slower, because they propagate out to more code. We not only have to refactor our logic, but the logic in all relevant tests as well. Some would argue that means the tests aren't written well or aren't testing at the right level. But writing tests at the right level is hard. It can take some time before we can learn how to write effective tests, just like it takes time before we can learn how to write effective code.

But the biggest concern I have with agile development is that it can be used as an excuse for poor judgement. Being lazy is not agile, it's being lazy. Always taking the easy way out usually catches up to us, and then it can be painful to dig out of that hole.

Here are my takeaways:

  1. If we don't understand the problem, we can't process our way out of it. In other words, if we don't know what the issues really are, no amount of process can help us.
  2. We learn best from our own mistakes. This informs our future decisions much more so than any process could.
  3. Context matters most.

Addition by Subtraction

December 06, 2009

Generally speaking, when we think of adding value, we usually think of what we can add to make improvements. But another way we can make improvements is to remove what's not essential. By trimming out what's not needed, focus is shifted back towards what's important. In some cases, these subtractions can actually add up to a whole lot.

This perspective is usually directed towards design. And for good reason. Simpler designs are friendlier, easier to use, and generally yield better outcomes. But we don't always share this perspective when thinking about software. Frameworks, apis, even programming languages themselves can all benefit from simpler perspectives. Strangely enough though, complexity here is often touted as an asset instead of a fault. In fact, the implicit reasoning is usually that things need to be complicated for them to be worthwhile.

Sometimes we even inflate simpler problems into harder ones. This serves as an excuse to implement more complex solutions. Unfortunately we can do this without even realizing it. And the more abstract the original problem is, the more room there is to introduce complexities. In some cases, it might be easier to err on the side of solving none of the abstract problems instead of trying to solve them all.

So if we find ourselves several layers removed from the original problem, it might be time to take a step back and evaluate how we got here. It might be the right path, or it might not. But if it's the wrong path, then we should turn around.


August 26, 2009

We try to keep things simple. After all, nobody starts a project thinking, let's make this one complicated for a change. Yet somehow, things like to end up that way.

I like to think of this as a shock, but it's usually not that surprising, all things considered.

Perhaps the most obvious reason is that programmers knowingly introduce this complexity themselves. This complexity is justified by rationalizing that it will anticipate future situations, and account for them elegantly. The goal here is to prevent future maintenance headaches, but ironically it can add complexity if the wrong abstractions are made.

The opposite can also be just as true. As code design debt accumulates, not acting early enough once a problem is recognized can also lead to increased complexity.

Programmers can also have drastically different opinions on subjects. If we take a look around the software world, we'll see that there are many operating systems, langauges, frameworks, and a myriad of all kinds of tools. Each one of these represents a particular area where a programmer identified a gap, and then proceeded to make an attempt at filling it in. This naturally leads to overlap among similar software. This overlap itself isn't bad, in fact it's probably a little bit healthy. Each competitor can focus on what it feels is important, and users can choose accordingly. But merely having more anything can contribute to its own kind of complexity to deal with. After all, it's easier to make a choice given fewer options.

But along with the lack of consensus on what's important, programmers also don't think the same things are unimportant. Some are willing to let documentation drift out of sync. Some aren't interested in repeatable builds. How about automated tests? Commented code? Consistent style? Design patterns? Stable back ends? Intuitive front ends? Practically everyone will agree that it's important to have many of these items, but not everyone will agree on which one is ok to let slip. These differences can help contribute to software complexity as programmers fight against each other.

Programmer 1: I never knew it would be so hard to push a car downhill.
Programmer 2: What do you mean, downhill!?

There's also the idea that things aren't worthwhile unless they are complicated. When things are simple and easily understandable, it's easy to suggest avenues of improvement. But if too many distractions are entertained, the codebase can start developing warts that can be difficult to deal with.

It is easy to make things. It is hard to make things simple.