โFinding, accessing, and using data effectively is one of the biggest challenges companies will face.โ -- Janice Zdankus, VP of Innovation for Social Impact in HPEโs Office of the CTO.
Data holds great value, but only if you can find it and access it. For businesses today, data is stored in different formats, across different environments, and across many different platforms. Each of these entities operates as independent silos, which inhibits an organizationโs ability to harness value locked within the data.
HPEโs Ellen Friedman details this challenge in a recent CIO.com article, Getting the Data You Need: Connecting Data Producers and Data Consumers. The entire article is excellent, but for those who want a quick summary, read on.
Challenges of sharing the same data for different purposes
Friedman says that often data collected for one purpose may be valuable for other purposes within an organization. โCustomer transaction data, for instance, may have been collected for billing purposes, but it also provides a rich source of training data for online recommendation systems, fraud detection, and supply chain predictive analytics.โ She explains this is one reason why a business should run AI and analytics applications together on the same system. But if your system does not easily support data sharing, you may miss out on this add-on value.
To find and use the data you need, you first must know it exists. Data is typically generated by different groups; therefore, it often is siloed. When you find the data, you will need to be able to combine diverse data sets. Unfortunately, you may have mismatches in data type, granularity, or other parameters. Data discovery is often difficult due to the huge proliferation of different types and sources of data.
Data trustworthiness is also a fundamental challenge. Once you have access to the data, you will need to copy it to a secondary system and normalize it (you remove redundant files.) At best, this takes weeks, but typically it takes months. Due to this long lag time, the data becomes stale, which could render it unreliable.
Data accessibility is another challenge to data sharing. For example, the data infrastructure may not support broad accessibility by different types of applications or tools. Add to that the issue of when and where you will use data. Will you need to move large data sets, or can you access them remotely?
Making it easier to find, access, and share data
The need to share data is not new, but demand is growing. Because of this pressure, new data initiatives are being launched to provide better data sharing for large, distributed data sets.
One new approach is called Dataspaces. This is a new service platform being developed by HPE to help data producers and data consumers discover the data they need, integrate diverse data sets, and improve data governance and trust.
Friedman explains how technology innovation in the form of a unifying data infrastructure for large-scale, highly distributed systems is also helpful in sharing data.
Another approach to making data easier to find and use involves inserting appropriate metadata into large data sets. Suparna Bhattacharya, Principal Technologist with HPE AI Research, is developing new techniques for automating metadata insertion.
A new data initiative launched by the Linux Foundation, called the Agstack Foundation is building a free and open source digital infrastructure for agriculture stakeholders with the goal to improve global agriculture. HPE is a gold member of the Agstack Foundation.
Better data use and sharing
Finding, accessing, and using data effectively is a big challenge for businesses today. Yet, we are now seeing new developments and initiatives to solve this problem, along with a growing trend toward better data sharing. In the coming days, much of that siloed data can be set free, easily sharing it, and bringing its value to life.
Read Friedmanโs full article Getting the Data You Need: Connecting Data Producers and Data Consumers. To find out more watch the video โData Spaces: Connecting to Data You Can Trustโ. You can also read โGetting the most from your data-driven transformation: 10 key principlesโ Find out more about HPE Ezmeral File and Object Store software.
Hewlett Packard Enterprise
twitter.com/HPE_Ezmeral
linkedin.com/showcase/hpe-ezmeral
hpe.com/software
AlisonGolan
Alison Golan is a writer/editor for HPE's social marketing team. For 30+ years, sheโs been writing about technology โ from hardware and software to networking and streaming. She started her tech career as a public relations specialist, arranging media coverage with CBS, CNN, CNBC, The New York Times, The Wall Street Journal, Business Week, and Fortune. Today, she enjoys transforming technical jargon into compelling stories.