Transforming IT
Showing results for 
Search instead for 
Did you mean: 

Designing Your Big Data Infrastructure: A New Functional Architecture


Big Data architectures are driving the introduction of new models for designing and deploying infrastructure architectures in data centers. As adoption of Big Data increases, it will result in significant innovations in information and communication technology (ICT) infrastructures. If Big Data is about addressing new business models heavily based on new capabilities to manage information, then data center infrastructure architecture – network, storage and servers – has to evolve to support these new data stores and their movement requirements.


Even though today Big Data adoption tends to be based on a phased – “start small” – approach, organziations should carefully consider from the beginning how to connect new Big Data technologies to their existing infrastructure. The objective here is to plan for a smooth, service-oriented roadmap to such innovative architectures.


In any transformation, we should first ask what architecture works best for big data, given likely future needs, and then ask how we can combine that with the existing environment. And this will require models to simplify adoption. With our team of great technologists, HP has developed the following Big Data functional architecture that helps us simplify future transformation activities.


Big Data Functional Architecture Model




Big Data Processing

The elements of processing big data are shown in blue; Collection, Computation, and Consumption.



Collection is about capturing the data. Collection occurs from many different sources, including social media, business transactions, documents, the web and other places. We then move the data across the Internet or a private network to some form of storage, where the data is retained for some period of time.



Computation can be viewed from at least two perspectives; storage and processing. From a storage perspective, and depending on how data is categorized and what retention policies are required, we may be able to archive the data. Similarly we may be required to destroy the data because it has no further value or places us at risk. Further, some data will be transformed into the refinery system to add value.



Data is consumed through presentation of data. Presentation can take the form of reporting, visualization tools, or dashboards. New infrastructures, such as Hadoop or SAP HANA, may require the addition of new tools, applications, or infrastructures to consume data they hold. Consumption of data also occurs through other platforms; for example, Vertica or Autonomy may consume data from Hadoop.

Big Data Management

As soon as the data used by a company is stored in its IT environment, there is the technical and compliance need to manage the life cycle of such data. Because companies work on different data sets, it’s usual to find many different data life cycles, specific for each data set.


Being the collection and consolidation point of large heterogeneous data, the Big Data Refinery System leads to a new Data Lifecycle Management integrated approach, which is a key aspect of the Big Data solution.

Big Data Technology Integration

Big Data is based on a set of technologies and architectures designed so that enterprise organizations can extract value from very large volumes of a wide variety of data by enabling fast and reliable processing. Big Data requires a shift in computing, storage, and networking architecture so that IT can handle the data processing workloads and data management and integration required to transfer and analyze large volumes of data.


I would like to thank Donald Livengood, Luciano Boschetti and Tari Schreider, for their contributions to this architectural model. What do you think of this model? Drop a comment in the box below and let me know your opinion.


 And take a look at my previous blog, in which I wrote about the concept of the Big Data Refinery System, a key reference element in an IT transformation for Big Data.


Key Takeaways

Today, IT infrastructures that support data architecture need to evolve. They will need to accommodate new technologies. To understand the impact and plan for the integration and evolution of the data center, organizations need to have reference architectural models that enable them to drive the introduction of these new technologies.


Enterprises’ IT departments need to understand and plan for the architectural impact of new Big Data technologies and plan for the transformation of today’s infrastructure in order to provide Big Data services to the business.


See you in June, at HP Discover in Las Vegas, where we can have a deeper discussion on this topic.

Click here to learn how the HP Big Data Transformation Experience Workshop can help you build a roadmap for your Big Data infrastructure strategy, while reducing risk and accelerating decision-making.


To learn how I can help your organization meet its growth objectives, see my HP Technology Expert profile.


Discover 2012 banner.png

Amos Ferrari
0 Kudos
About the Author


As a Digital transformation strategist, I help companies, define and communicate these intended benefits as part of the vision for the change. As a Solution strategist, I lead agile creation of new consumption based products, from new business model creation to initial minimum viable product.