Tech Insights
Showing results for 
Search instead for 
Did you mean: 

When Big Data analytics needs to happen at the edge


by Eddy Biesemans

Global Account Manager


A decade ago, when the concept of big data analytics began to gain steam, attention turned to banking, equities, insurance, advertising, biopharma, and other business and scientific fields that generated lots of digital data. Analysis could take place in the data center or in the cloud, giving companies valuable insights into customer behavior, product performance, and research. 

Few anticipated the exponential growth of data from industrial IoT systems. Now, just like their counterparts in business and scientific fields, managers in manufacturing, oil & gas, energy grids, smart cities, transportation, and agriculture want to derive actionable insights using big data analytics. But unlike white-collar fields, industrial big data can’t be compiled and analyzed in data centers or in the cloud. It has to take place on the edge of the network, right where the action is. 

Big data analytics use case: self-driving cars 

HPE works closely with engineers, data architects, and customers on projects ranging from cyber security to smart grids. We also have an ongoing project for a client in the automotive industry that uses big data analytics on the edge. 

This company is developing a new autonomous self-driving car. The testing regimen requires some 22 million hours of driving time to evaluate and analyze multiple automotive systems in a wide variety of conditions. If you calculate that forward, it comes to approximately 2,500 years! No one can wait that long, of course. 

The technology we have jointly deployed with Schneider Electric is a big data analytics appliance. There are 40 of them collocated in locations around the world where testing is taking place. Instead of one car driving 22 million hours, there are thousands of vehicles driving around for thousands of hours each. The cars have IoT sensors monitoring speed, road conditions, and nearby objects such as pedestrians, curbs, and other vehicles. When the test units reach one of the 40 collection points or branch offices, IoT data collected during the trial is transmitted to the appliance. The data is automatically processed and analyzed at the edge, with the results sent back to headquarters. 

Why do it this way? Consider the alternative: If the big data packages from the trials had to be transmitted to the data center or cloud, it would be nearly impossible.

We’re talking about thousands of test drives every day, generating many petabytes of data. To make that much bandwidth available to transmit all of it would be prohibitively expensive, and prone to latency and timing issues. 

Instead of bringing data to the analytics applications at the data center or cloud, the edge appliances bring analytics to the data as it’s being gathered. The analytical models run on every one of the 40 appliances, enabling real-time processing at the edge. Only the results are uploaded to HQ, avoiding all of the problems with latency, timing, and availability. Critically, this approach contributes to a much faster time to market. 

Running big data analytics on the edge is applicable to other industries as well. Basically, any use case that requires big data on the edge is a candidate. We are looking at oil and gas, energy grids, and other industrial scenarios in which lots of IoT big data needs to be processed on the plant floor or out in the field.

Join HPE at Industry of Things World in Berlin on September 23-25 to learn more about using data for the manufacturing process - particularly the concept of closed-loop manufacturing - and applying AI in the industrial setting to make sense of this data. Get a preview of the agenda here.

Intel logo.png


HPE Edgeline Converged Edge Systems is powered by Intel® Xeon®.

Empowering the Digital Enterprise to be more efficient and innovative through data-driven insights from the Internet of Things (IoT)
0 Kudos
About the Author


Starting June 23
HPE Discover Virtual Experience
Joins us for HPE Discover Virtual Experience live and on-demand
Read more
Online Expert Days - 2020
Visit this forum and get the schedules for online Expert Days where you can talk to HPE product experts, R&D and support team members and get answers...
Read more
View all