AI-Driven Insights Through Enterprise Big Data Platforms

Abstract

Artificial Intelligence (AI) and enterprise big data platforms are emerging as a key aspect in today's digital transformation, changing the way enterprises are able to utilise the huge and diverse data they produce. This review summarizes the state of the art of the use of AI techniques, specifically machine learning techniques and deep learning techniques, within scalable big data architectures, and the ability to derive actionable insights across a variety of industries including healthcare, finance, manufacturing and retail. This is provided against the backdrop of a number of pivotal research studies, and we review the analytical tools used to analyse enterprise data, as well as some practical examples of the benefits they can bring. We suggest an integrated theoretical model that we call AI-Driven Enterprise Insight (ADEI), with three interdependent layers of support, namely the data foundation, the intelligence engine, and the value realization layer, interconnected by a governance envelope, and a continuous feedback loop. The literature-based synthesis of benchmarks shows a strong consistency between the performance of the AI-based platforms and that of traditional analytics, with AI-based platforms being consistently more accurate, more throughput, and faster to insight than traditional analytics, and exhibiting a persistent balance between predictive power and interpretability. Another focus of the review is on the challenges at the intersection of data quality, scalability, privacy, security, and explainability of insights produced by AI. Finally, we discuss some potential future avenues of research and provide a unified view of today's field and its future.