Are Random Events the Key to Successful Innovation?

Unfortunately, this article from The Econmist is available to subscribers only. But it is really interesting, so at the risk of getting in trouble, here is the good part:

In his book "Why Innovation Fails"*, Carl Franklin quotes an intriguing study by three academics, Donald Lehmann at Columbia University in New York, and Jacob Goldenberg and David Mazursky both at the Hebrew University of Jerusalem. The researchers looked at 197 product innovations, of which 111 were successes and 86 failures. What they found was that the successful innovations had some, or all, of the following features: they were moderately new to the market, based on tried and tested technology, saved money, met customers' needs and supported existing practices. By contrast, the products that failed were based on cutting-edge or untested technology, followed a "me-too" approach, or were created with no clearly defined solution in mind.

So far, so obvious. What was much less so was the researchers' list of "idea factors"-where the ideas for the innovations came from, and how they determined the success or otherwise of the ensuing innovation. For instance, "need spotting" involved actively looking for an answer to a known problem, while "solution spotting" meant finding a new way of using an existing piece of technology-much as the CD player capitalised on the recently invented laser diode. "Mental inventions" were things dreamed up in the head with little reference to the outside world, while "random events" were serendipitous moments when innovators stumbled on something they were not looking for but immediately recognised its significance. The two other sources of ideas were "market research" and "trend following".

When the team plotted the success-to-failure rate of the six different idea factors, the two worst by far turned out to be trend following and mental inventions. Both produced three times as many failures as successes. By contrast, need spotting produced twice as many successes as failures. Market research generated four times more, and solution spotting seven times more successes than failures. But the clear winner in the innovation stakes was "taking advantage of random events", which generated 13 times more successes than failures.

This is why companies like Google let employees spend work time on non-work things. Time spent doing non-work related tasks may lead to more of these random events. I know it seems counterintuitive – less time spent *working* could lead to more profit via more innovation – but in many industries it is true.