How to Assess AI-Generated Work, Evidence, and Claims

Using artificial intelligence (AI) to draft this introduction might have been faster than writing it from scratch, but by the time a human expert checks for accuracy, rewords awkward phrases, and adds context the AI couldn’t know about, it might take more time to write with AI than without it.

That conundrum sits at the heart of our latest issue of ThinkSet—and today’s increasingly AI-powered economy. As the authors of a recent MIT Sloan research paper put it, “most economic models still treat AI as a labor substitute … implicitly assuming that cheap machine output translates directly into realized value.” This paradigm is “dangerously incomplete” due to one binding constraint: human verification bandwidth, or “the scarce capacity to validate outcomes, audit behavior, and underwrite meaning and responsibility when execution is abundant.”

This is a new version of an age-old problem. In 1300, England’s King Edward I—worried that goldsmiths were committing fraud—ordered that no silver or gold be sold until “guardians of the craft” had tested it and stamped it with a leopard’s head.

More recently—as Jason Gu, former head of AI at Visa, notes in one of this issue’s Q&As—the credit card industry faced a similar problem. Gu argues that its solution should be applied to clearing AI-assisted work: agree in advance what counts as good enough to proceed, what doesn’t, and who bears the loss when an error gets through.

Errors may lead to costly litigation or enforcement actions. That’s important to keep in mind from the start of an organization’s AI journey. As Brad Wilson, Eric Matrejek, Alex Jacobs, Kelli Foy, Steve Chapin, and Zach Skrehot write, both building and buying AI infrastructure involve tradeoffs if a dispute occurs. Relatedly, Chau Hoang and Carl Losito discuss what makes an AI-assisted valuation defensible.

AI requires more due diligence to ensure bad actors aren’t using the technology to commit fraud—a particular concern for financial institutions. Barbara Vanich, former chief auditor and director of professional standards at the Public Company Accounting Oversight Board, outlines questions audit committees should ask about AI-enabled fraud.

Meanwhile, in healthcare—where margins are increasingly thin—a key challenge involves verifying AI’s return on investment. Patrick Higley and James McHugh offer insights into how health systems can prove their AI is working by measuring the right outcomes.

The verification economy extends beyond AI. Ceren Canal Aruoba and Kristina Shampanier explain how legal teams, amid a growing number of digital privacy disputes, should evaluate the question at the heart of these cases: What did a consumer notice and understand about a particular privacy disclosure?

We round out this issue with a high-level explainer and infographic on the verification economy covering critical risks, what it means for general counsel and executives, and best practices organizations can use to prove what they claim.

In an economy where claims are cheap, proof and expertise are the new premiums. The organizations that focus on verification now are the ones others will trust later.

We hope this new issue of ThinkSet, penned by our experts across AI, healthcare, economics, and investigations, will help business leaders do just that.