Measuring AI Success: A Maturity Framework for Digital Business Transformation

Abstract

Artificial intelligence (AI) has become a central driver of digital business transformation, yet a persistent gap remains between organizations' AI investments and the value they realize. Despite the role of AI in digital business transformation, there is a huge disconnect in the value that organizations are deriving from their investments in AI and how much they are spending on AI. Many AI programs do not go beyond pilot programs, and the current methods of measuring IT success, which are based on traditional IT measures, are not well aligned with measuring the multidimensional, dynamic nature of AI-induced transformation. This review aims to overcome this by collating available literature on how to measure the success of AI and organizational maturity, and by suggesting that an integrated maturity framework could be used to measure the success of AI in digital business transformation. The framework draws on the resource-based view and dynamic capabilities theory, and identifies five dimensions of capabilities that form a continuum of interdependent capabilities (strategies and vision, data and technology, people and culture, governance and ethics, process integration), expressed as a multidimensional maturity assessment, which is explicitly connected in the framework to measurable transformation outcomes, through a feedback loop that transitions from initial to optimizing. To validate, 175 organizations were analyzed with the results providing preliminary empirical support: maturity stage was strongly and positively correlated with value realization and reported ROI – together with the other dimensions explaining 61% of the variance in value realization; the other dimensions (people, culture and governance) emerged as the dominant factors – of which people was most important – with the other dimensions explaining an additional 39% of the variance in value realization, in addition to the complementarity effect observed at similar maturity levels across different capability profiles. The review also provides a unified conceptual frame and a practical guide to diagnosis and prescription for practitioners as well as an agenda for future research focusing on longitudinal validation, standardisation of measurement, and incorporation of new AI paradigms.