Big Data or Big Buzz? It's hard to read an IT publication without "Big Data" splashed across the cover. However, there remain three important Big Data realizations for CIOs that deserve special mention.
1. Data Quality Matters
Let’s face it, you take a lot of pain to run ETL (Extract, Transform, and Load) processes before you load data into BI systems, and it’s important that the same (or some) discipline be applied to Big Data before it is placed alongside traditional data in your BI systems.
Before you embark boldly into a Big Data project, it's important to institute data retention, classification, and archival policies, just as you did the first time around. If at all, these practices are even more relevant given the data volumes and growth with which you are dealing.
2. Unstructured data is most valuable when analyzed in the context of structured data
There's no doubt that unstructured data contains forward-looking insights that we cannot mine from structured data alone which is typically backward looking. Even so, most people roll their eyes when they hear unstructured data because it is likely that they are thinking about it separately from structured data. The sweet spot is at the cusp i.e. when unstructured data is analyzed alongside structured data (think product managers making pricing or design decisions based on past sales and current social media sentiment).
Like with any BI project, you need to first understand which data is relevant to your enterprise. This could get tricky with unstructured data because given the noisy nature of unstructured data, the more data you process, the more statistically significant (and therefore, reliable) your insights will be.
3. Big Data is a matter of When, not If
Business leaders have been wooed by Big Data. Better insights from data are a top priority for execs, with an overwhelming 89% saying they want better insights through business intelligence and analytics. With good reason too. One study indicates that top performing companies in almost every business sector were five times more likely to be leveraging analytics.
In some sense, the last decade could be described as a distributed era of BI that is about to give way to the rise of a mission-critical era of BI that will transform BI into a true enterprise-class application, fueled by Big Data. The easiest way to think about Big Data is not as a different animal but as a valuable extension and complement to your existing BI systems, which is really the essence of this blog post.
We recently recorded two podcasts on this topic with the amazing Claudia Imhoff at the Boulder BI Brain Trust - you can find links to the podcasts on the top right corner under Related Links.
In the video below, Silicon Angle experts, John Furrier and Dave Vellante, discuss some of the finer points on Big Data at HP Discover 2012.