IT and data scientists: Creating an environment for success

[AlisonGolan]

How IT can keep the enterprise running smoothly, while supporting experimentation and creativity for data scientists.

When my kids were young, I struggled daily to keep our household organized and running smoothly. My three kids, on the other hand, had a very different goal. They wanted to explore their creativity with as many toys as possible.

This constant push/pull resulted in finding a happy medium. I would let the house get a bit messy as long as our organized chaos still allowed us to function well as a family.

A recent article published in CIO.com, reminded me of how businesses must also create this type of balance. Big data analytics expert at HPE, Matt Maccaux, details the interesting relationship between IT and data scientists in his article, What IT Must Do to Make Data Scientists Happy.

I recommend you read the article in its entirety. But for those who want to read the Cliff Notes version, here you go.

IT vs data scientists: a lesson in contrasting goals

Maccaux explains how the work of data scientists is highly dependent on others — especially IT. Yet, typical IT operations don’t naturally support the work of data scientists.

Think of a business as a factory, where there’s a relationship between the people who create the products and those who are running the factory,” says Maccaux. “ In many ways, IT is like those in charge of running a stable factory. Data scientists, by contrast, are both coming up with new ways to make the factory run better and products for the factory to make and then push out to the market.

• IT’s processes and goals:

IT craves predictability and is focused on running mission-critical systems in a stable and reliable way. They generally follow standardized processes, methodologies, systems, and tools. IT also relies on automation, as they can’t function using manual processes in unique environments. In general, IT wants to operate in fixed time frames and in a predictable manner that manages changes in an orderly way.

• Data scientists’ processes and goals:

The core value of data science is about innovation. Their goal is to examine data and come up with new insights that can help run the business better. Data science is agile and spontaneous, seeking to go where the data leads. Each problem and each scientist is singular, and every problem has a unique set of data and tools required to solve that problem. Therefore, they are always seeking new tools, techniques, algorithms, and research that are incorporated into their work.

Creating balance: the optimal IT-data science relationship

To ensure data scientists succeed in their goals, IT should provide then with the raw materials and capabilities to do their jobs, as well as testing their prototypes. Because there’s an inherent tension between these two roles, the ideal relationship is when data science has a baseline of capabilities from IT, but IT also creates limits to prevent unnecessary risks

“Once the products are proven and reliable, data scientists want their work to become part of the factory, hand off their oversight to IT, and not have to babysit these projects anymore,” continues Maccaux. “This allows data scientists to continue to do what they do best: experiment and innovate.”

IT should create on-ramps and off-ramps for data scientists

To enable an optimal IT-data science relationship, Maccaux explains how IT needs to create on-ramps to support work environments for data scientists. That means IT should provide data products incorporating all data that are usable, accurate, and unify all sources of data into one or more product, people, or customer objects. Ideally, data scientists can then create new purpose-built data sets to drive innovation. IT must also provide an environment where the data science team can operate at all levels of an organization’s data stack and bring in new data when necessary. 

Once data scientists succeed with an innovation, IT must provide an off-ramp from the lab so the models and analytics (and the data supply chains that feed them) can be passed off to the IT team to run. The two groups must work together to establish plans for converting the successful innovation into a production environment. Meanwhile, data scientists must keep in mind the practical implications of how their creations can be mainstreamed and brought to market.

IT should help data scientists avoid common pain points in their work, including unnecessarily onerous data prep, making it simple to find and prepare data (often through data catalogues), and finding ways to test and support data ops. Additionally, IT can aid data scientists by ensuring they have the computing power they need to build models.

IT can also support data science is by working collaboratively in an R&D-like fashion where the production process never stops.

“When data scientists come up with new tools, IT can start the validation of the tool even before the product is ready for production," continues Maccaux. "IT must have a data science production factory that can accept new algorithms for productizing and bringing to operational maturity, with all the required resiliency, compliance, and other factors.”

Maccaux explains this type of support speeds the iterative process by allowing data scientists to focus on creating new algorithms instead of building the infrastructure to put those algorithms to use. By providing the on- and off-ramps that empower data scientists to do their work, IT can develop an ideal relationship with data scientists to create a thriving production cycle for the business. 

A smooth enterprise and happy data scientists

In the end, it’s the job of IT job to make sure the factory floor runs efficiently, whereas the job of data scientists is to push the boundaries. Working together, both teams can create an ideal environment – one that supports both stability and creativity.

Read the full article: What IT Must Do to Make Data Scientists Happy. To further explore data science best practices and how to adopt a framework that maximizes business productivity and accelerates time to value, check out the new IDC white paper: Industrializing Data Science with Data Analytics Factory Framework (DAF).

Alison

Hewlett Packard Enterprise

twitter.com/HPE_Ezmeral
linkedin.com/showcase/hpe-ezmeral
hpe.com/software