Enterprise Services
Showing results for 
Search instead for 
Do you mean 

Unknown knowns and Big Data

‎03-28-2014 08:57 AM - edited ‎09-30-2015 07:02 AM

By:  Vilas Prabhu, Enterprise Architect, Hewlett Packard Company


exclamation point amidst question marks.jpgLet me tell you a story.


Ten years ago, BuildALot construction engineers did a successful build of a specialized railway bridge suitable for a specific weather condition in Alberta province of Canada. Now their EMEA division is bidding for a similar project in Kazakhstan. Sally in the sales department vaguely remembers the weather condition details. She recalls reading about it in a case study during her sales training when she joined BuildALot seven years back. However she cannot place the case study nor can she find anyone who knows about that project. The chief engineer for the Alberta project still works for BuildALot Canada. In fact he is considered an authority on such projects. He is happily unaware of the Kazakhstan bid and value that he can add to that bid. All the documents related to the project are archived as it was older than seven years, so a search on weather condition via the corporate intranet does not yield any result. The BuildALot EMEA team is now struggling to put together a bid of sufficient quality as they lack the requisite knowledge.


Turnkey project developers (such as BuildALot), medium to large scale services organisations and public sector organisations all have this issue. They don't know enough about themselves. Sometimes their customers, suppliers, service users know more about them than the organisation itself.


It is the issue of "unknown knowns".


This issue can cause serious problems for organisations. It can make or break the business. It can cause problems in addressing new business opportunities, in handling customer service, in dealing with other stakeholders. The inability to quickly identify the “go to” person, where they are located, or what tool or methodology they used - can cost the organisation dearly.


This issue arises due to a lack of access to knowledge. Organisations try many tricks to address this issue. They train their people to impart their specific knowledge. They make collaboration tools available and hope knowledge gets acquired through them. Sometimes an informal network of information brokers develops to address this issue. But none of these approaches address the issue in its entirety.


We must understand the reasons behind this issue to come up with a comprehensive solution.

The main reasons for the inaccessibility of knowledge are: 

  • Size and spread of oraganisation.  The organizational spread across geographies, number of personnel and years of operations create enormous volumes of useful data. This volume is one of the reasons for inaccessibility of knowledge. 
  • Churn in personnel.  There is constant change in personnel in such organisations. People come and go, and roles change. This creates variability in data as each person brings different styles and degrees of completeness of data creation. 
  • Churn in product, services, rules, regulations etc.  Organisations are constantly changing their products, services, policies, rules, and regulations. This constant change adds a kind of velocity to the data pool. The problem is compounded when coupling this velocity with volume. 

Any solution must distill knowledge from the underlying data which exhibits these properties. Otherwise the solution won’t solve the problem in its entirety. Apart from traditional approaches such as specialised training, collaboration tools and information brokering via humans; we must look for other approaches.


One such approach is enabled by Big Data, owing to volume, velocity and variability challenges posed by underlying data.  Using Big Data technologies, such as Autonomy IDOL, it is possible to construct Knowledge Graphs out of all unstructured data that an organisation possesses including archived content. These knowledge graphs can then be efficiently queried to locate the desired information. These knowledge graphs can also be upgraded and improved based on experience of its usage, again using big data technologies.


Wouldn't BuildALot be a more effective organisation if they employed big data technologies to keep their knowledge accessible to people who need it, when they need it?


Previous blogs by Vilas Prabhu:

Related links: 

  • Big Data Solutions                                                                                                                            

About the Author


Vilas Prabhu - cropped.pngVilas Prabhu, Enterprise Architect, Hewlett Packard Company

Vilas is a technology thought leader. He was instrumental in development and implementation of a Model Driven Architecture toolset for many large program engagements.  Vilas currently promotes architecture-led approaches for Enterprise’s Information Systems transformation and sustenance, specifically in the area of analytics and data management.

0 Kudos
About the Author


Nov 29 - Dec 1
Discover 2016 London
Learn how to thrive in a world of digital transformation at our biggest event of the year, Discover 2016 London, November 29 - December 1.
Read more
Each Month in 2016
Software Expert Days - 2016
Join us online to talk directly with our Software experts during online Expert Days. Find information here about past, current, and upcoming Expert Da...
Read more
View all