I have noticed that as people age, they become finer and finer versions of themselves. Their eccentricities become sharper and more pronounced; their opinions and ideas more pointed and immutable; their thoughts more focussed. In short, I like to say that they become more perfect versions of themselves. We see it in our friends and acquaintances and in our parents and grandparents. It seems a part of natural human development.

Back in 2006, Netflix initiated the Netflix Prize with the intent of encouraging development of improvements in the accuracy of predictions about how much someone is going to enjoy a movie based on their movie preferences and rewarding the winner with $1,000,000. Contestants were given access to a set of Netflix’s end-users’ movie ratings and were challenged to provide recommendations of other movies to watch that bested Netflix’s own recommendation engine. BellKor’s Pragmatic Chaos team was announced as the winner in 2009 having manage to improve Netflix’s recommendations by 10% and walked off with the prize money.

What did they do? Basically, they algorithmically determined and identified movies that were exceptionally similar to the ones that were already liked by a specific user and offered those movies as recommended viewing. And they did it really well.

In essence what the Bellkor team did was build a better echo chamber. Every viewer is analyzed, their taste detailed and then the algorithm perfects that taste and hones it to a razor sharp edge. You become, say, an expert in light romantic comedies with a strong female lead, who lives in a spacious apartment in Manhattan, includes many dog owners, no visible children and often features panoramic views of Central Park.

Of course, therein lies the rub. A multifaceted rub at that. As recommendation engines become more accurate and discerning of individual tastes they remove any element of chance, randomness or error that might serve to introduce new experiences, genres or even products into you life. You become a more perfect version of you. But in that perfection you are also stunted. You are shielded from experimentation and breadth of experience. You pick a single pond and overfish it.

There are many reasons why this is bad and we see it reflected, most obviously, in our political discourse where our interactions with opposing viewpoints are limited to exchanges of taunts (as opposed to conversations) followed by a quick retreat to the comfort of our well-constructed echo chambers of choice where our already perfected views are nurtured and reinforced.

But it also has other ramifications. If we come to know what people like to such a degree then innovation outside safe and well-known boundaries might be discouraged. If Netflix knows that 90% of its subscribers like action/adventure films with a male hero and lots of explosions why would they bother investing in a story about a broken family being held together by a sullen beekeeper. If retail recommendations hew toward what you are most likely to buy – how can markets of unrelated products be expanded? How can individual tastes be extended and deepened?

Extending that – why would anyone risk investment in or development of something new and radically different if the recommendation engine models cannot justify it. How can the leap be made from Zero to One – as Peter Theil described – in a society, market or investment environment in which the recommendation data is not present and does not justify it?

There are a number of possible answers. One might be that “gut instincts” need to continue to play a role in innovation and development and investment and that risk aversion has no place in making the giant leaps that technology builds upon and needs in order to thrive.

A more geeky answer is that big data isn’t yet big enough and that recommendation engines aren’t yet smart enough. A good recommendation engine will not just reinforce your prejudicial tastes, it will also often challenge and extend them and that we don’t yet have the modelling right to do that effectively.  The data are there but we don’t yet know how to mine it correctly to broaden rather than narrow our horizons. This broadening – when properly implemented – will widen markets and opportunities and increase revenue.

Everyone who is anyone loves bandying about the name of Clayton Christensen, the famed Professor of Business Administration at the Harvard Business School, who is regarded as one of the world’s top experts on innovation and growth and who is most famous for coining the term “disruptive innovation“. Briefly, the classical meaning of the term is as follows. A company, usually a large one, focuses on serving the high end, high margin part of their business and in doing so they provide an opening at the low end, low margin market segment.  This allows for small nimble, hungry innovators to get a foothold in the market by providing cheap but good enough products to the low end who are otherwise forsaken by the large company who is only willing to provide high priced, over-featured products.  These small innovators use their foothold to innovate further upmarket providing products of increasingly better functionality at lower cost that the Big Boys at the high end.  The Big Boys are happy with this because those lower margin products are a lot of effort for little payback and “The Market” rewards them handsomely for doing incremental innovation at the high end and maintaining high margins.  In the fullness of time, the little scrappy innovators disrupt the market with cheaper, better and more innovative solutions and products that catch up to and eclipse the offerings of the Big Boys, catching them off guard and the once large corporations, with their fat margins, become small meaningless boutique firms.  Thus the market is disrupted and the once regal and large companies, even though they followed all the appropriate rules dictated by “The Market”, falter and die.

Examples of this sort of evolution are many.  The Japanese automobile manufacturers used this sort of approach to disrupt the large American manufacturers in the 70s and 80s; the same with Minicomputers versus Mainframes and then PCs versus Minicomputers; to name but a few.  But when you think about it, sometimes disruption comes “from above”.  Consider the iPod.  Remember when Apple introduced their first music player?  They weren’t the first-to-market as there were literally tens of MP3 players available.  They certainly weren’t the cheapest as about 80% of the portable players had a price point well-below Apple’s $499 MSRP.  The iPod did have more features than most other players available and was in many ways more sophisticated – but $499?   This iPod was more expensive, more featured, higher priced, had more space on it for storage than anyone could ever imagine needing and had bigger margins than any other similar device on the market. And it was a huge hit.  (I personally think that the disruptive part was iTunes that made downloading music safe, legal and cheap at a time when the RIAA was making headlines by suing ordinary folks for thousands of dollars for illegal music downloads – but enough about me.)  From the iPod, Apple went on to innovate a few iPod variants, the iPhone and the iPad as well as incorporating some of the acquired knowledge into the Mac.

And now, I think, another similarly modeled innovation is upon us.  Consider Tesla Motors.  Starting with the now-discontinued Roadster – a super high end luxury 2 seater sport vehicle that was wholly impractical and basically a plaything for the 1%.  But it was a great platform to collect data and learn about batteries, charging, performance, efficiency, design, use and utility.  Then the Model S that, while still quite expensive, brought that price within reach of perhaps the 2% or even the 3%.   In Northern California, for instance, Tesla S cars populate the roadways seemingly with the regularity of VW Beetles.  Of course, part of what makes them seem so common is that their generic luxury car styling makes them nearly indistinguishable, at first glace, from a Lexus, Jaguar, Infiniti, Maserati, Mercedes Benz, BMW and the like. The choice of styling is perhaps yet another avenue of innovation.  Unlike, the Toyota Prius whose iconic design became a “vector” sending a message to even the casual observer about the driver and perhaps the driver’s social and environmental concerns.  The message of the Tesla’s generic luxury car design to the casual observer merely seems to be “I’m rich – but if you want to learn more about me – you better take a closer look”. Yet even attracting this small market segment, Tesla was able to announce profitability for the first time.

With their third generation vehicle, Tesla promises to reduce their selling price by 40% over the current Model S .  This would bring the base price to about $30,000 which is within the average selling price of new cars in the United States.  Even without the lower priced vehicle available, Tesla is being richly rewarded by The Market thanks to a good product (some might say great), some profitability, excellent and savvy PR and lots and lots of promise of a bright future.

But it is the iPod model all over again. Tesla is serving the high end and selling top-of-the-line technology.  They are developing their technology within a framework that is bound mostly by innovation and their ability to innovate and not by cost or selling price.  They are also acting in a segment of the market that is not really well-contested (high-end luxury electric cars).  This gives them freedom from the pressures of competition and schedules – which gives them an opportunity to get things right rather than rushing out ‘something’ to appease the market.  And with their success in that market, they are turning around and using what they have learned to figure out how to build the same thing (or a similar thing) cheaper and more efficiently to bring the experience to the masses (think: iPod to Nano to Shuffle).  They will also be able thusly to ease their way into competing at the lower end with the Nissan Leaf, Chevy Volt, the Fiat 500e and the like.

Maybe the pathway to innovation really is from the high-end down to mass production?

I have recently been “chosen” to receive a fistful of invitations to Google‘s newest permanent beta product Google Wave.

This new application is bundled along with an 81 minute video that explains what it is and what it does. My first impression upon noticing that little fact suggested that anything that requires almost an hour and a half to explain is not for the faint of heart. Nor is it likely to interest the casual user. I have spent some time futzing around with Google Wave and believe that I am, indeed, ready to share my initial impressions.

First, I will save you 81 minutes of your life and give you my less than 200 word description of Google Wave. Google Wave is an on-line collaboration application that allows you to collect all information from all sources associated with the topic under discussion in one place. That includes search results, text files, media files, drawings, voicemail, maps, email, reports…everything you can implement, store or view on a computer. Additionally, Google Wave allows you to include and exclude people from the collaboration as the discussion progresses and evolves. And in the usual Google manner, a developer’s API is provided so that interested companies or individuals can contribute functionality or customize installations to suit their needs.

Additionally, (and perhaps cynically) Google Wave serves as a platform for Google to vacuum up and analyze more information about you and your peers and collaborators to be able to serve you more accurately targeted advertisements – which, after all, is what Google’s primary business is all about.

All right…so what about it? Was using Google Wave a transformative experience? Has it turned collaboration on its head? Will this be the platform to transform the global workforce into a seamless, well-oiled machine functioning at high efficiency regardless of geographical location?

My sense is that Google Wave is good but not great. The crushing weight of its complexity means that the casual user (i.e., most people) will never be able to (or, more precisely, never want to) experience the full capabilities of Google Wave. Like Microsoft Word, you will end up with 80% of the users using 20% of the functionality with this huge reservoir of provided functionality never being touched. In fact, in a completely non-scientific series of discussions with end-users, most perceive Google Wave to be no more than yet another email tool (albeit a complex one) and therefore really completely without benefit to them.

My personal experience is that it is a cool collaboration environment and I appreciate its flexibility although I have not yet attempted to develop any custom applications for it. I do like the idea of collecting all discussion-associated data in one place and being able to include appropriate people in the thread and having everything they need to come up-to-speed within easy reach. Personally, I still need to talk to people and see them face-to-face but I appreciate the repository/notebook/library/archive functionality afforded by Google Wave.

I still have a few invitations left so if you want to experience the wave yourself and be your own judge, post a comment with your email address and I’ll shoot an invite out to you.