Generative AI’s toughest question: What’s it worth?

Over the past couple of years, companies have deployed generative artificial intelligence (AI) for everything from customer relationships to software coding to product design. The percentage of businesses using generative AI increased from 33% in 2023 to 65% in 2024, according to a global McKinsey survey of nearly 1,400 respondents. But beneath the excitement lies an important question for finance leaders: What’s the return on all this investment?

Finance leaders are planted at this intersection of enthusiasm, cautious optimism, and skepticism, tasked with helping companies come up with useful business metrics and stay ahead of the curve without wasting money. That task becomes all the more important as the technology matures. The management consultancy Gartner suggested that generative AI is entering a new phase of an emerging technology’s so-called “hype cycle.” After an initial wave of “inflated expectations,” the technology is steaming into the “trough of disillusionment.”

That means businesses are getting down to the nitty-gritty details of the “use cases that drive ROI,” according to Gartner. This process is increasing the pressure to prove that investments in generative AI are more than hype and flash — and skepticism is growing in some quarters.

BALANCING GENERATIVE AI’S VALUE AND COST

A Gartner survey of more than 600 business leaders worldwide highly involved in generative AI found that nearly half of them (49%) considered estimating and demonstrating generative AI’s business value the biggest hurdle to adoption. “As organizations scale AI, they need to consider the total cost of ownership of their projects, as well as the wide spectrum of benefits beyond productivity improvement,” Leinar Ramos, senior director analyst at Gartner, said of the survey results.

“There’s a … shift toward wanting some degree of certainty. The biggest issue for me is around the cost, and keeping a handle on the cost,” said Craig Johnson, ACMA, CGMA, strategic development director at Agena Group in the UK. “Anyone who has ever done any kind of infrastructure project will know that the potential for cost overrun is incredible.”

EARLY SUCCESSES

Some companies have reported early successes with generative AI and other new technologies. One example is Klarna, a Swedish fintech company that said its use of generative AI helped the company reduce its first quarter 2024 spending on sales and marketing by 11%, the equivalent of $10 million a year in annualized savings. Most of Klarna’s savings came from using generative AI tools like Midjourney and DALL-E, which the company claims have slashed its “image development cycle” from weeks to days. Having quicker and cheaper creative processes makes it easier to market around holidays, the company said.

“Traditionally, it would have been very costly to cater to these occasions with bespoke imagery, but with AI that is no longer an issue, and we can create relevant imagery to support practically any event. Essentially, we have removed the need for stock imagery,” the company’s chief marketing officer, David Sandström, said in a news release.

HOW TO MEASURE AI’S BENEFITS

Translating early successes into useful business metrics is a different story. Some respondents in the McKinsey survey estimated cost savings or revenue increases. Most respondents saw cost decreases in human resources. The area where most respondents projected meaningful revenue increases (greater than 5%) was with supply chain and inventory management. But those are only estimates, and how to quantify and track the benefits of a generative AI project is hotly debated.

Measuring benefits and ROI on information technology adoption has always been challenging. One reason is the long-term nature of gains. Benefits can be cumulative and accrue over an extended period, making it difficult to measure immediate returns. Another reason is that benefits are often intangible or qualitative in nature. It is difficult to quantify intangible benefits such as enhanced employee productivity, better decision-making, better customer experiences, and innovation enablement. In addition, IT projects can often be interdependent with other projects and systems. This makes it challenging to isolate the specific impact of one project on overall performance.

Teaming together, finance and information technology professionals have a key role to play in partnering with leaders across the business to establish a disciplined and measured approach to adopting AI technologies, according to Paul Parks, CPA, CGMA, director of Management Accounting–Americas for AICPA & CIMA. This starts with a road map that provides steps toward adopting generative AI, Parks said. The road map should include the establishment of project governance, data governance, assuring employees are properly trained, updating acceptable use policies, and developing an AI adoption policy. It will be important to focus resources and projects in a targeted way to where they will provide the highest level of return. According to Parks, calculating return on investment and project benefits should include multiple steps:

• Define objectives: Identify the use case and what management aims to achieve with the generative AI investment. Business goals can include cost savings, revenue generation, and/ or productivity improvements.

  
  

• Estimate costs: Capture all costs associated with the project, including development, implementation, and ongoing incremental operational costs.

  
  

• Estimate financial and nonfinancial benefits: While intangible benefits can be hard to quantify, over time they will contribute to organizational success.

  
  

• Regularly review and adjust: Monitor project performance and adjust project direction, as necessary.

“Generative AI is evolving rapidly, and the power of this technology has the potential to unlock new possibilities in automation, enhance decision-making, and drive innovation to new levels,” Parks said. “The finance organization has an opportunity to be an advocate and enabler of AI adoption. By providing a disciplined approach to adoption and working closely with business leaders in allocating resources in a strategic manner, benefits can be transformative.”

WHAT COMPLICATES TRACKING AI COSTS

No matter the methodology, tracking generative AI value is an involved endeavor, Agena Group’s Johnson said. Simply predicting costs may be more difficult than with other technology endeavors because generative AI relies on intensive computing power and comes with.

“The cost structure, and the concept of cost by query, can be radically different,” Johnson said.

Decisions on generative AI costs are also complicated by the pace of change. A bespoke implementation that is costly today may become much cheaper as technology providers develop solutions. “If we try and build and spend money fixing something now, chances are [technology companies] are going to fix it in time,” Johnson said.

On the other side of the equation, measuring AI’s benefits can be maddening, especially in large organizations. “The benefit side is inherently difficult to measure, because most organizations are running multiple projects in parallel,” Johnson said. Too many leaders, he warned, “get very, very bogged down in how to measure the benefits, when actually it’s an arbitrary thing. The important thing is to deliver a program of change.”

PICKING A PROBLEM

The keystone for determining ROI in a new AI project may be defining the business problem that needs to be solved in the first place. “If we are going to launch a new product using emerging technologies, the priority will always be how with the use of this technology we are solving the current business problem for our customer offering a better value proposition in terms of product, service, and costs while being cognizant of your environment, your infrastructure, from top to bottom,” said Muhammad Asif, FCMA, CGMA, CEO of PrimeFour Consulting in Dubai.

The framing of the business problem, and devising the right solution, should account for impacts on business agility and profitability, Asif continued.  “It’s how your product or business process innovation is relevant to your organization’s operations and strategy — how you are envisioning your long-term growth,” he said. Creating a shared vision for transformation may be the responsibility of everyone involved in the project. But ultimately, finance will have a special role in balancing the excitement of the new endeavor with the realities of the budget.

“There has to be a leadership commitment … of being an organization that does try to make changes, but also keeping a handle on cost,” Johnson said. “You need to be the champions of that way of working.” The keystone for determining ROI in a new AI project may be defining the business problem that needs to be solved in the first place.

Dr. Lovett has 30+ years experience in the accounting and finance fields. He is a noted author, columnist, speaker, and contributor to the financial success of multiple businesses and nonprofit organizations. Dr. Lovett can be contacted at dr.lovett@fl-business-consultants.com.