We’ve got to be at or near the Peak of Inflated Expectations in the Hype Cycle for Big Data.  It’s the point where the meme seems so powerful that everyone wants to associate themselves with it.

But, as happened with data mining, unstructured data mining, and other fevered dreams of extracting ponies from the manure heap of raw data, what if the insights we all believe are lurking in our data… aren’t lurking, or can’t be lured out of hiding?

I ran across a couple of posts this week that bear on the issue.

A post from Jeff Jonas. who can always be relied on to smash false idols, deals with this question.  As Jonas says:

The problem being; often the business objectives (e.g., finding a bomb) are simply not possible given the proposed observation space (data sources).

Dan Woods re-posts another variation on this theme:

…the data created and maintained outside your company is becoming much more important than the data that you can acquire from internal sources. Yet, few companies realize this and fewer are taking action. Instead, they are suffering from the Data Not Invented Here Syndrome.

In other words, there’s a difference between Big Data techniques and magic.  Sigh.

Your thoughts?