Big Data
Showing results for 
Search instead for 
Do you mean 

Analytics for Human Information: New Top Ten Myths of Big Data - Myth #6

ChrisSurdak ‎11-14-2013 05:00 AM - edited ‎02-19-2015 01:38 PM

As I mentioned in Myth #5, many organizations feel particularly challenged by the design and deployment of their Big Data environment.  Indeed, the technical challenges are not trivial. To succeed in this initial step you have to bring together a team of people with a wide range of skills and experience.  If you are one of the few who have built a working analytics environment, congratulations. But now you are probably looking for more people with even rarer skills and experience.


As more organizations jump the technical hurdles of Big Data they find themselves addressing the need for so-called “Data Scientists”…people who can actually define and drive the analytics process and begin to generate genuinely new insights from the massive quantities of data at hand. This, in turn, is feeding another new myth of Big Data, which brings me to the topic of this discussion.


Big Data Myth #6: Analysis is the Hardest Part of Big Data

Last week, I emphasized that in the long(er) run “Data Scientists” will be more important to Big Data success than “Hadoop Rock Stars.” I acknowledge the challenges of the technology and in properly architecting Big Data solutions but after this exercise is over, the work of actually analyzing the data begins. We need to crank up the analytics in order to learn from our data.


However, the analysis of the data and the generation of new insights may be just as barren of business value if we do not take the critical next step, which is to act upon those insights.


I cannot stress this enough; the point of Big Data is not to learn about Open Source Software.  Nor is it to gain lots of new knowledge about our businesses or our customers.  The point of all of this effort and hoopla is to act differently. To change what our organizations do, how they do it, and why they do it. To not reach this end goal means that all prior efforts were merely an academic exercise.  I built a cool system and I learned some fun facts that I can discuss at a cocktail party.  If I don’t act on those facts then I might as well have piled up all of the cash I just spent and lit it with a match.


That being said, there is yet another group of people who may rapidly supplant the analysts.  To my knowledge no one has yet coined a term for these people, but whoever does is likely to rake in the accolades. Put simply, these people are process engineers, although we might refer to them as fulfillment scientists, resultologists, operational transmorgrifiers (as in the cartoon Calvin and Hobbes) or some similarly exotic label. In any event, this group of people will be tasked with taking the insights generated by data scientists and actually doing something with that information.


Now, you might be thinking to yourself “Great. It’s important to act on the insights. Got it.”  On its face, such action doesn’t sound like a big deal, but I promise you it truly is.  The idea of Big Data is that I can take massive amounts of data and process it so that I can understand my thousands or millions of customers on an intimate, one-on-one, minute-by-minute basis.  Such understanding means that you can find out what each customer wants or needs right now, in this instant, and give it to them.  If you meet that micro-market, you can charge more because you delivered more value.  You make more money, increase customer satisfaction and all kinds of other pleasant effects that we’re all searching for.


But, if the rest of your organization is not prepared for this you will almost certainly screw this up.  Great, you know what your customer wants this very second, can you act on that?  Can you act at the speed of insight?  If your answer is ‘no’ (and unless you grew up as, it likely is ‘no’) then you are heading for a deep canyon of disillusionment. 

You might know what each customer wants, every second of the day, but your present processes almost certainly cannot respond that fast as they were never designed to operate at this sort of speed or volume.  They will break and when they do, your employees will become frustrated and your customers will simply move on. 


I know that this sounds rather dire, but it is the reality that many organizations will face if they do not prepare themselves for action at the speed of insight. The process revolution that I’m referring to is the elephant in the room that people aren’t acknowledging, because they’re still early in the adoption curve for these technologies.  But, that means that the opportunity, the imperative, to act on this issue is now. Before the need arises. 


So, as much as I emphasized in Myth #5 that analytics is extremely important to making Big Data pay off, this pending process revolution is the final, critical step in the transformation of value chains.  If the goal is the monetization of Big Data, this last step is the one giant leap that will get you there…or kill you while trying.


Now, many of us that lived through the process reengineering era of the 1990’s aren’t really thrilled with the prospect of going through all of that again.  Tough!  It won’t be easy this time either, and the stakes will be much higher now because of how dramatically compressed business cycles have become over the last 20 years. Having said that, there is a potential solution to this issue, or at least a method for reducing the pain involved. BUT, that’s the topic for next week’s post, and I’m not ready to go into it yet.


See, you wanted an answer and I wasn’t ready to give it to you… Frustrating, isn’t it? 


Click below to continue reading about The New Top Ten Myths of Big Data



0 Kudos
About the Author


Chris Surdak is a Subject Matter Expert on Information Governance, analytics and eDiscovery for HP Autonomy. He has over 20 years of consulting and technology experience, and holds a Juris Doctor from Taft University, an MS from the Wharton School at the University of Pennsylvania, a CISSP Master's Certificate from Villanova and a BS in Mechanical Engineering from Penn State. Chris is author of the Big Data strategy book, "Data Crush," which was recently nominated as International Book of the Year for 2014, by GetAbstract. Chris is also contributing editor and columnist for European Business Review magazine.

John Cousineau
on ‎11-16-2013 06:57 AM

Chris: thanks for this. Intrigued by your notion of the "emergence of fulfillment scientists, resultologists, operational transmorgrifiers [love the terms] tasked with taking the insights generated by data scientists and actually doing something with that information". When we get 'big data' and 'analytics' right, I'm wondering: might we just be talking about front-line employees? - John

27 Feb - 2 March 2017
Barcelona | Fira Gran Via
Mobile World Congress 2017
Hewlett Packard Enterprise at Mobile World Congress 2017, Barcelona | Fira Gran Via Location: Hall 3, Booth 3E11
Read more
Each Month in 2017
Software Expert Days - 2017
Join us online to talk directly with our Software experts during online Expert Days. Find information here about past, current, and upcoming Expert Da...
Read more
View all