Digital Transformation
Showing results for 
Search instead for 
Did you mean: 

How Digital Disruptors use Data Science


In “the old days”, data analysts analysed data for business managers, creating monthly or, for fast moving businesses, weekly reports.

How things have changed! Firstly, data analysts are now called Data Scientists, and they get paid a lot more. And secondly, data science has become one of the key weapons of digital disruptors. 


15.big data.png


Ways in which digital disruptors use data science 

There are three ways in which digital disruptors use data science

A . To provide product functionality, like Uber’s surge pricing or Airbnb’s pricing recommendations

B. To develop better products, more quickly and more accurately 

C. To offer data back to their customers, like the APIs offered onto road data by the Swedish Transport Authority


A. Data Science in the product

Rather than being used to analyse how a product sold or manufactured, data science is now being used to provide product functionality. Let’s look at some examples:

Predictive maintenance : the latest General Electric locomotive is leased on a “pay per use” basis. It is therefore in GE’s interest that its locomotives are always running. In order to ensure this, the trains are packed with sensors. The data from these sensors is then put thru a predictive maintenance systems, predicting when maintenance is required, rather than waiting for a breakdown to occur. HPE does the same with its Gen8 and Gen9 servers. If you allow them, HPE support will remotely collect the telemetrics from your server, put this data thru a prediction engine and give advice on pro-active server maintenance. 

Prediction in general : of course, predictive maintenance is just one of example of prediction. There are many more, not least in the insurance industry. Data science is improving all the time. In a recent article, Andrew McFee explains how a website called Kaggle allows competitors to pit their prediction algorithms against each other. AllState insurance put data on this web site, and the winner of the prediction algorithm competition was 3.5 times better at prediction than AllState’s own algorithms. 

I recently visited Nottingham Trent University (NTU) in the UK. NTU uses seven different “digital footprints” from students to predict when they may need help from their tutors. These prediction algorithms are able to predict the need for help about 6 to 8 weeks ahead of when a tutor might typically realise their help was need. Because NTU sees its product as “helping students reach their full potential”, these prediction algorithms are very part of their product. 




Micro-charging and leasing : the GE locomotive I introduced above is not just an example of predictive maintenance, it’s also an example of micro-charging. Rather than flat rate charging, companies are looking to charge on a micro-basis. For example, there are now smart refuse bins that charge you every time you put trash into them. Insurance companies are looking to use telemetrics from cars to micro-charge for car insurance, which varies depending upon where you drive and how you drive. Truck manufacturers are looking to do similarly, charging depending upon “thrust-hours”. Gartner believes that micro-leasing will become the norm for rental of any expensive manufacturing plant. 

Other examples of precision : We’ve already talked about precision charging. Precision, driven by higher resolution digitization, will fuel other types of digital transformation. With precision farming, a 4K picture is taken using a drone. This is then automatically analysed, using data science, to determine where to fertilise, where to apply pesticide, where to water, and in the UK, where to apply slug killer. In the future, these precision applications of farming chemicals will be done by a robot. Precision medicine is another area. In the future, we can expect customised anti-cancer treatments, for example. 

15.farming robot.jpg


Recommendations : When you put your space onto Airbnb, it makes a pricing recommendation. This recommendation is not pulled from thin air. It is in fact the output from a very clever recommendation engine that takes a whole series of variable into consideration. 

Netflix likewise; their “what shall I watch next?” recommendation engine is a closely guarded secret. Spotify, the music streaming service, makes “similar music” recommendations, “new releases this week you may like” recommendations; and even “shall I create a playlist of music you like to listen to in the evenings or when running” recommendations.

Recommendation engines are being used by “financial services digital disruptors”. Zopa was, apparently, the world’s first peer to peer lending network. In a recent interview Zopa’s CEO, Giles Andrews, gave on BBC News, he explained that the heart of his product was a very sophisticated risk engine; essentially a prediction engine applied to the loaning of money. 

Dynamic pricing : No discussion on digital disruptors is complete without at least one mention of Uber. In fact, I have two. Firstly, then demand for rides far outstrips supply, Uber puts in place “surge pricing”. Uber uses data science to create a market that “gets reluctant drivers out of bed” - Uber is using data science to “market create”, much like the stock market does. Uber is also moving into ride sharing. In order to do this, it needs to determine who might share a ride with who. Uber uses prediction to do this. It knows, to about 70% accuracy, that if you want a ride from a certain place at a certain time of day, where you will going to.


B. Developing better products, faster

Digital disruptors are fast and relentless. They are constantly releasing new functionality. They try things - they experiment. In order to do this, digital disruptors need a feedback loop. They use the data from their customers’ use of their product to get fast and accurate feedback on how these products are being used.

Lets look at examples of how they do this using data science.

On race days, Nascar analyses all the tweets and fan site activity relating to the race. It uses data science to bucket this human interaction data into topics, and for each topic it automatically determines sentiment. It then addresses any concerns thru information on its fan site, information at the event, or feeds to it broadcast partner, Fox Broadcasting. What Nascar is doing is using data science to form a fast feedback loop to dynamically adjust its “product” - it’s product being both the race day, but also the information it provides around the race. 




When HP Inc releases a new printer, they monitor up to 200 web sites. Like Nascar, they bucket the feedback, determining sentiment and product issues for each bucket. Yes, HP Inc could wait for support calls to come in. But social media-based feedback is a faster and more “honest” feedback mechanism. Also, research shows that digital natives rarely take the official support route, preferring instead to tweet their frustrations. 

That’s two examples where human interaction data is used as a feedback loop data source. Because of big data techniques, we can also use machine data. Almost always, machine data is copious, and often comes at us in floods. Before the advent of technologies like HPE’s Vertica, conventional data analysis platforms were simply overloaded. Not any more.

Game Show Network,, is part of Sony’s gaming division. When they create a new game for, say, the iPhone, they record the terabytes of “touch stream” machine data coming from the game. This tells them which areas customer like; what they don’t like; areas they never visit; and areas they get thru too quickly (either because the game is too easy, or because a cheat has been posted on the web). This machine data-based feedback loop allows to quickly and accurately determine when they need to change for the next version of the game. So fast is this feedback loop that can go from new product to next version of the product in just seven days, should it be necessary. 

Both Twitter and Yammer are keen experimenters. They will release two or three different version of a new interface and then use the machine data from customers using these interfaces to figure out which version is most effective. 

Criteo is a company that recommends ads based upon your browsing or email content. It uses state-of-the-art machine learning algorithms to make these ad recommendations. These algorithms are their product. Criteo uses machine data analysis to determine the effectiveness of this product. If they adjust the product, they can tell, within minutes how effective that change has been. 


C. “Build the APIs and they will come”

I was in Sweden a month ago. I was talking to some customers and they told me how the Swedish Road Authority had created APIs into the data they possessed. I thought I’d get more details on this. I found out that there in fact 73 transport authorities who offer APIs onto their data!

John Deere offers data gathered from customers’ farms back to them thru




Digital disruptors know that the data they hold is useful. Rather than figure out exactly how people might use this data, they are offering access to it thru APIs. 

If you are planning to become a digital disruptor, either of business models or of your business operations, consider offering APIs onto the data you hold.



Digital disruptors do a lot more than use data to provide monthly business reports.

They use advanced data science techniques to provide product functionality from prediction, to dynamic pricing, to recommendations to precision pricing and precision resource usage. 

They also use human interaction data and machine data to create a very fast and accurate feedback loop from the release of their latest product to the design and build of the next version. Some digital disruptors can go around this loop in a matter of hours.

And finally, some digital disruptors offer data back to the customers thru APIs.





Mike Shaw
Director Strategic Marketing

linkedin.gifMike Shaw

0 Kudos
About the Author


Mike has been with HPE for 30 years. Half of that time was in research and development, mainly as an architect. The other 15 years has been spent in product management, product marketing, and now, strategic marketing. .

Jan 30-31, 2018
Expert Days - 2018
Visit this forum and get the schedules for online HPE Expert Days where you can talk to HPE product experts, R&D and support team members and get answ...
Read more
See posts for dates
HPE Webinars - 2018
Find out about this year's live broadcasts and on-demand webinars.
Read more
View all