We've all had our frustrating moments with computers. We bang our heads against the walls for quite some time, and no matter what we try, the computer responds with a clever "I thought you might try that, here's your error." And then, we try talking to someone else about it, and they usually have a brilliant idea that solves everything elegantly. Then we're left scratching our heads, wondering why we didn't think of that before.
Lots of problems get solved like this simply because they are looked at from a fresh perspective. It's easy to get lost in the details of a problem. When we keep our heads down, it's hard to realize that we were just approaching the problem from the wrong angle.
Programming is a game of insight.
I think that sums up the essence of programming. The most significant gains are often those that shed the problem in a new light.
But not all reevaluations of a problem lead to successes. In fact, I would argue that most of them don't. But the ones that do work, usually do so in a big way. Given that programming is this give and take process, progress often isn't linear. There isn't a lot of progress, or it looks like things are getting worse, and then suddenly, there's a big jump.
This makes it especially difficult to measure, assuming it's even possible to measure at all. Any formal attempts at measurements results in programmers optimizing for the local maxima. This ends up detracting from productivity.
But regardless of whether we can keep track of it or not, it's important to foster an environment that encourages creativity. A single idea can change everything.