Artificial Intelligence Pitfalls - Introduction

[Mattab]

AI Pitfalls

At my opening keynote of the AI Summit London this year I reflected on the evolution of Artificial Intelligence over the past few years. Back in 2017, I was doing a lot of education around what AI is, how it works, and how the data science discipline operates. Wind forward to 2022 and AI has progressed in leaps and bounds.

However, this has come at a cost; AI does share some similarity with other IT projects but there are several unique challenges. Indeed, despite a foundation of sound academic principles, in 2019 87% of data science projects were failing. As a mainstream technology that has become a cornerstone of digital transformation initiatives, this figure is pivotal.

A lot of lessons have been learnt and there has been an explosion of techniques and tools (more on the challenges these present in a later article). As a relatively new discipline facing several similar challenges to traditional software development, as well as many specific to the discipline, successful adoption is about learning, adapting, and implementing best practice.

Through this blog series, my aim is to shine a light on common AI pitfalls; observations that have culminated from customer engagements, academic research, peers at HPE, and from being inquisitive about what HPE customers and partners are doing. There are many things that can hamper AI projects from initiatives that are interesting science projects to abstract dreams. Wherever you are on the AI journey, however, this series should provide some guidance on the practicalities of getting AI right. The following areas will be explored over seven articles:

Traditional vs AI projects – There are several aspects to building out an AI that are common across traditional approaches to software development. There are plenty of well documented challenges when it comes to engineering software solutions, so it is important to evaluate traditional software development and explore the challenges that are unique to AI.

Design decisions – These will have a big influence on the types of skills needed. Understanding the ramifications of the build, buy and hybrid design decisions is paramount to ensure that the right technology is being brought in to tackle the right problem. All too often I encounter the sledgehammer being used to crack a nut!

Skills – Building team AI is complex. AI is a broad discipline with multi-agent collaboration as a key component. This multi-disciplinary team needs to comprise of not just Data Scientists but Data Engineers, Business Analysts, Operational staff, Systems Architects, as well as Executive sponsors and DevOps Engineers.

Culture – Naive extrapolation or lack of understanding can result in AI projects facing an uphill battle to get buy-in and acceptance. Building this into the project right at the start is paramount.

What is the right pattern? – There are many architectural anti-patterns, or ways to get it wrong. There are, however many many ways to get it right. Making the right design decisions and choosing the right architectural patterns are key enablers to not only successfully implement but also scale.

The problem of scale – The Data Science Day Two challenge is one of scale; moving from small, often local, implementations to large integrated systems is not without problems. To get models incorporated into a business process that can scale linearly is the final step in moving models from conception to realisation.

Remember AI is a journey not a destination because AI operates with a fluid substance, data. Data is changing and evolving all the time, which means AI models need to adapt to this shifting landscape otherwise their value diminishes.

With the right help and guidance, you can be on a road well-travelled with lessons learnt and clear risk mitigation identified. There are two ways you can approach AI: the hard way! Going on the journey, learning the lessons through trial and error. or the easy way engaging with an organisation like HPE, that has the experience necessary to make deployments both successful and timely.