AI Agent ROI Is a Discipline, Not a Dashboard

Last week I closed the worldview column with a promise and a warning. The promise was that this week we would talk about AI agent ROI, the actual return on investment from the agents arriving on campus. The warning was to bring your CFO. So I went hunting for the number that makes that meeting easy: a named institution, an AI agent from a major campus platform, a measured outcome. I read the launch releases and the momentum decks.

The number is not there. Not hidden. Not disputed. Absent.

So let me give you the direct answer this title owes you, because the absence is the argument. Measuring AI agent ROI takes four things a campus controls: a countable unit of finished work, defined in writing before the demo; a baseline recorded before the agent turns on; outcome metrics instead of activity metrics; and a cost ledger that counts integration, oversight, and maintenance alongside the subscription. No platform ships those four things. Not even the one I run. They are decisions, and the return on investment conversation is coming to your cabinet whether or not you have a number to bring.

The Embedded AI Wave Has Invoices. The Worker Agents Have Receipts.

Here is what the hunt turned up. When the biggest SIS vendor in our market launched its AI-native student platform this spring, the release described a knowledge graph of nearly ten thousand higher education workflows. It named zero adopting institutions and zero measured outcomes, and the trade press said so in plain terms. The other suite vendors run the same play: catalogs, adoption counts, quadrant placements. Workday can tell Wall Street its AI products are approaching six hundred million dollars in annual recurring revenue. A real number, but a number about what institutions are spending, not about what they are getting back. Invoices, not receipts.

Now look at where the receipts live, because they exist. Georgia State put its mascot chatbot through randomized controlled trials: admitted students nudged by Pounce enrolled at a 3.3 point higher rate, cutting summer melt by about a fifth, and first-generation students getting course-embedded messages scored about eleven points higher on finals. The University of Hawaiʻi published operational numbers on its named campus bots this year: nineteen hundred student questions answered without a human touching them, 165 staff hours returned, ninety-four percent of students opting in. Notice the shape: a narrow agent, a boring and countable job, an institution willing to publish the measurement. I said recently that I intend to keep our characters boring. Add the corollary: boring jobs are where ROI is measurable. Nobody can measure the return on "AI across your entire enterprise." Anybody can measure deflected password tickets.

I have two predictions on the record that say the embedded suite AI wave will underperform the cross-application agents. Both are still on the clock, and the vendors' revenue growth is a counter-signal I will keep naming: renewal-exposed dollars are a form of customer testimony. But willingness to pay is not evidence of value received. The receipts I am waiting for have an institution's name on them, and the next natural place to look is the sector's big annual conference stage this fall.

Your CFO Already Knows Nobody Is Measuring

While you were reading vendor decks, your CFO was reading the survey data on AI in higher education, and it is brutal. When Inside Higher Ed polled college chief business officers this summer, ten percent reported measurable return on AI investment. One in ten. A veteran CBO quoted in the coverage doubted even that, saying they found it hard to believe those campuses could actually calculate the ROI. The EDUCAUSE workforce research puts a floor under that suspicion: thirteen percent of institutions measure AI ROI at all, and most of that thirteen percent admit they are not sure how. Half of campus technology leaders told the same survey ecosystem this spring that AI's return is unclear or below expectations.

Here is the line that should make the whole cabinet sit up: the spending is rising anyway. Half of those same technology leaders now rank generative AI investment as a high or essential priority, and nearly every education organization surveyed this year expects its AI budget to grow or hold. Forrester predicts this is the year CEOs pull CFOs into approving AI investments directly, because fewer than a third of enterprises can tie AI value to profit-and-loss changes. Budgets up. Measurement absent. Enrollment down.

That gap is not a technology failure. It is a governance failure from a familiar family: capability adopted before ownership is assigned. The campus that deployed agents without naming an owner for each one has a cousin, and it is the campus that funded agents without an owner for the number. Walk into the CFO's office with a measurement plan before the budget hearing walks into you.

The Numbers Everyone Quotes Fail the Same Audit

Before you build your AI agent ROI plan, clear the folklore off the table, because the numbers in circulation fail in both directions.

Start with one my own industry loves. Identity vendors, mine included, have spent years repeating that forty percent of helpdesk calls are password resets and that each reset costs seventy dollars, with big analyst names attached. Chase those claims to the source and they dissolve. The real ancestor is an analyst research note from a quarter century ago that said resets were ten to thirty percent of calls, with a cost range wide enough to announce itself as an assumption stack. The seventy-dollar figure points at a paywalled report whose own public abstract contains no such number; vendors have simply been citing each other in a circle. My team traced both statistics and cut them from print. This paragraph is why.

Now the other direction, because the doom numbers fail the same audit. The most quoted AI statistic of the past year, that ninety-five percent of enterprise AI pilots produce zero return, comes from a working paper, not a peer-reviewed study. Its own limitations section calls the figures directionally accurate and interview-based, and by the paper's own funnel roughly a quarter of actual pilots succeeded. The famous customer-service triumph runs the same way: a payments company announced its assistant did the work of seven hundred agents and estimated forty million dollars in profit improvement. Nobody outside the company audited either number, and a year later its own CEO said the cost focus went too far and quality suffered. The vendor scoreboards are no cleaner. One major support-AI vendor advertises an average resolution rate of seventy-six percent, with a definition that counts a customer walking away as a resolution. Independent tests of the same product measured thirty-eight to fifty-three.

The lesson is not that everyone is lying. The lesson is that this category runs on three words that get swapped as if they were synonyms. Deflected. Contained. Resolved. A deflection rate counts questions that never reached your queue, which includes the student who gave up. A containment rate counts conversations that never escalated, which includes the student who quit arguing with the bot at midnight. A resolution rate counts work that actually got done and stayed done. Only one of those belongs in a business case, and any metric that can go up while your students' experience gets worse cannot defend an investment. That is the test. Run every deck through it.

The J-Curve Is an Argument for Measuring, Not for Waiting

Now the strongest case against my own skepticism, taken straight. Real evidence of AI agent value exists. A top economics journal tracked the staggered rollout of an AI assistant across five thousand customer support agents: fifteen percent more issues resolved per hour, with the newest agents gaining the most. Salesforce told investors, on an earnings call where misstatements carry legal consequences, that its support agent has passed five million conversations, sixty-four percent resolved autonomously. And economic history gives my skepticism a genuine warning. Economists spent the late eighties asking why computers showed up everywhere except the productivity statistics; the payoff arrived years later, after firms reorganized around the technology. Electric motors took decades to move factory productivity. The literature now calls this the productivity J-curve: general-purpose technologies dip before they deliver. ERP itself showed a documented dip-then-payoff pattern. If CIOs in 1999 had demanded first-year profit proof, nobody would have bought the platforms this column spends its life integrating.

The historians may well be right that today's flat enterprise AI returns are the bottom of a J-curve. Here is what that argument does not license: skipping the measurement. You cannot see a J-curve you never instrumented; the curve is visible only to the organizations that baselined, counted, and kept the ledger honest through the dip. The support-AI industry itself just conceded the point, with one vendor now billing only for resolutions that survive three days without the ticket reopening. That is what a countable unit looks like: contractual, auditable, reopen-tested. Demand that shape from every vendor, and from the humans too: the most sobering experiment of this cycle found experienced developers nineteen percent slower with AI while sincerely believing they were twenty percent faster. Self-reported time savings are not a baseline. They are a mood.

Now the same audit, pointed at my own company. At QuickLaunch, the measurement surface is the part of the platform I would show your CFO: execution monitoring that reports every integration workflow's successes, failures, throughput, and retries by workflow and node, dashboards for the unglamorous countables like unused-but-live accounts and SMS spend, and an agent you can question in plain language. What no product of ours computes is your ROI, because the baseline and the counterfactual are governance decisions, not features. And the case numbers on our own site, like the university that cut IT workload by seventy percent, are exactly the class of vendor-reported number this column just taught you to interrogate. Interrogate ours. Ask for the workflow logs. Credibility is the brand, and I would rather lose a deal than lose that.

The working session fits on one page. Pick the boring, high-volume job with a countable unit. Write the unit down, reopen-test and all. Record the baseline before go-live, not after. Measure resolutions, never deflections. Count the whole cost: integration, oversight, maintenance, and the subscription. Then publish the number to your cabinet, even when it embarrasses the project. That is AI agent ROI as a discipline, and the CIO who runs it walks into every budget meeting owning the conversation. Next week the ledger flips to the liability column: the PowerShell scripts your identity operation has been quietly carrying for years, and why it is time to pay that debt down.

Take the Next Step

Primary: Put your measurement plan on real plumbing: the QuickLaunch Reporting and Analytics layer, shows every workflow's outcomes, and the iPaaS Agentic Design Studio is where the boring, countable integration agents get built. Current capability detail lives in the release notes

Secondary: Follow the weekly column on the QuickLaunch blog. NNext week: why it is time to remove the PowerShell script tech debt.

FAQ

How do you measure ROI from AI agents?

Measuring AI agent ROI requires four elements defined by the institution, not the vendor: a countable unit of completed work (such as a ticket resolved by the agent with no human involvement that stays closed for 72 hours), a baseline of cost and volume recorded before deployment, outcome metrics rather than activity metrics, and a fully loaded cost ledger covering integration, human oversight, maintenance, and subscription fees. The ROI is the change against baseline in the countable unit, priced against that full ledger.

What is the difference between deflection rate, containment rate, and resolution rate?

Deflection rate counts questions that never reached the human queue, including users who gave up. Containment rate counts conversations the AI never escalated, including users who quit without an answer. Resolution rate counts cases where the work was actually completed and stayed completed. Resolution is the only one of the three that belongs in an AI agent business case, and the strongest definitions are reopen-tested: a case counts as resolved only if the ticket does not reopen within a defined window.

Do universities see ROI from AI agents today?

Mostly not yet, by their own account. In 2026 surveys, only about 10 percent of college chief business officers reported measurable return on AI investment, and EDUCAUSE research found only 13 percent of institutions measure AI ROI at all. The documented positive results in higher education come from narrow, well-instrumented agents: Georgia State's Pounce chatbot raised enrollment 3.3 percentage points in a randomized trial, and the University of Hawaiʻi reported 1,900+ student questions answered without staff involvement and 165 staff hours saved in one academic year.

Is it too early to demand ROI from AI agents?

Economic history says returns from general-purpose technologies often arrive late, after organizations redesign work around them; researchers call this the productivity J-curve, and it held for computers, electrification, and ERP systems. That is a reason not to cancel promising agents at the first flat quarter. It is not a reason to skip measurement, because the J-curve is only visible to institutions that recorded a baseline and tracked a countable unit from day one.

References

• Ellucian Student Arrives — A Unified AI Platform Built for the Future of Higher Education* — PR Newswire (Ellucian). https://www.prnewswire.com/news-releases/ellucian-student-arrives--a-unified-ai-platform-built-for-the-future-of-higher-education-302747987.html
• Ellucian bets the future of higher education runs on agentic AI with launch of Ellucian Student* — EdTech Innovation Hub. https://www.edtechinnovationhub.com/news/ellucian-bets-the-future-of-higher-education-runs-on-agentic-ai-with-launch-of-ellucian-student
• Workday Q3 fiscal 2026 earnings call transcript* — The Motley Fool. https://www.fool.com/earnings/call-transcripts/2025/11/25/workday-wday-q3-2026-earnings-call-transcript/
• How an Artificially Intelligent Virtual Assistant Helps Students Navigate the Road to College* — Page & Gehlbach, AERA Open. https://files.eric.ed.gov/fulltext/EJ1194134.pdf
• Classroom Chatbot Improves Student Performance, Study Says* — Georgia State University News. https://news.gsu.edu/2022/03/21/classroom-chatbot-improves-student-performance-study-says/
• UH AI chatbots drive early intervention* — University of Hawaiʻi News. https://www.hawaii.edu/news/2026/04/07/uh-ai-chatbots-drive-early-intervention/
• College CBOs Sick of 'Just Plugging Holes'* — Inside Higher Ed (2026 Survey of Chief Business Officers). https://www.insidehighered.com/news/business/financial-health/2026/07/16/college-cbos-sick-just-plugging-holes
• Most Higher Ed Professionals Are Using AI at Work, New Survey Finds* — NACUBO, on EDUCAUSE's The Impact of AI on Work in Higher Education. https://www.nacubo.org/News/2026/1/Most-Higher-Ed-Professionals-Are-Using-AI-At-Work-New-Survey-Finds
• Half of Campus Tech Leaders Question AI's ROI* — Inside Higher Ed (2026 Survey of Campus Chief Technology Officers). https://www.insidehighered.com/news/tech-innovation/artificial-intelligence/2026/05/12/half-campus-tech-leaders-question-ais-roi
• AI Budgets in Education Show No Sign of Decline* — Campus Technology. https://campustechnology.com/articles/2026/06/01/ai-budgets-in-education-show-no-sign-of-decline.aspx
• Predictions 2026: AI Moves From Hype to Hard Hat Work* — Forrester. https://www.forrester.com/blogs/predictions-2026-ai-moves-from-hype-to-hard-hat-work
• Password Reset: Self-Service That You Will Love (Gartner Research Note T-15-6454, 2002), transcribed excerpt* — PasswordResearch.com. http://passwordresearch.com/stats/study76.html
• Best Practices: Selecting, Deploying, And Managing Enterprise Password Managers (public abstract)* — Forrester. https://www.forrester.com/report/best-practices-selecting-deploying-and-managing-enterprise-password-managers/RES139333
• The GenAI Divide: State of AI in Business 2025* — MIT Project NANDA (working paper). https://cloudelligent.com/wp-content/uploads/2026/02/v0.1_State_of_AI_in_Business_2025_Report.pdf
• Myth: MIT showed that 95% of AI pilots fail* — NewMR. https://newmr.org/blog/myth-number-2-mit-showed-that-95-of-ai-pilots-fail/
• Klarna AI assistant handles two-thirds of customer service chats in its first month* — Klarna. https://www.klarna.com/international/press/klarna-ai-assistant-handles-two-thirds-of-customer-service-chats-in-its-first-month/
• Klarna reinvests in human talent for customer service after AI chatbot push* — Customer Experience Dive. https://www.customerexperiencedive.com/news/klarna-reinvests-human-talent-customer-service-AI-chatbot/747586/
• Fin AI agent resolution rate claims* — Intercom (Fin.ai). https://fin.ai/
• Intercom Fin AI review: independent resolution testing* — Builts.ai. https://builts.ai/blog/intercom-fin-ai-review/
• AI contact center ROI metrics that actually defend the investment* — UJET. https://ujet.cx/blog/ai-contact-center-roi-metrics
• Generative AI at Work* — Brynjolfsson, Li & Raymond, The Quarterly Journal of Economics. https://academic.oup.com/qje/article/140/2/889/7990658
• Salesforce Q2 fiscal 2027 earnings call transcript* — The Motley Fool. https://www.fool.com/earnings/call-transcripts/2026/08/31/salesforce-crm-q2-2027-earnings-call-transcript/
• The Productivity Paradox of Information Technology* — Brynjolfsson, Communications of the ACM. https://dl.acm.org/doi/10.1145/163298.163309
• The Dynamo and the Computer: An Historical Perspective on the Modern Productivity Paradox* — Paul A. David, American Economic Review. https://ideas.repec.org/a/aea/aecrev/v80y1990i2p355-61.html
• The Productivity J-Curve: How Intangibles Complement General Purpose Technologies* — Brynjolfsson, Rock & Syverson, AEJ: Macroeconomics. https://www.aeaweb.org/articles?id=10.1257%2Fmac.20180386
• Investment in Enterprise Resource Planning: Business Impact and Productivity Measures* — Hitt, Wu & Zhou, Journal of Management Information Systems. https://www.scheller.gatech.edu/directory/research/information-technology-management/wu/pdf/hitt_wu_zhou_jmis_2002.pdf
• Zendesk introduces outcome-based pricing for automated resolutions* — Zendesk. https://www.zendesk.com/newsroom/articles/zendesk-outcome-based-pricing/
• Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity* — METR. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
• QuickLaunch Release Notes* — QuickLaunch. https://quicklaunch.io/resources/quicklaunch/

**Raymond Todd Blackwood is the President of QuickLaunch and writes about identity, agentic AI, and the messy reality of higher-ed IT. #ItsExistential*