Why Most Organizations Are Getting AI Adoption Wrong
Most organizations are asking the wrong question about AI. They want to know whether the model is good enough. The sharper question is whether the worker knows when they are not. A new Management Science peer-reviewed study by Andrew Caplin and colleagues makes that point with unusual clarity: AI helps most when people have an accurate read on their own limits. In other words, the future of human performance may depend less on raw talent than on calibrated judgment.
In the experiment, 732 participants judged whether people in photos were over age 21, sometimes with an AI confidence score and sometimes without it. The headline result is easy to grasp and hard to ignore. AI improved average accuracy, and it helped lower-ability participants more than stronger ones. But the decisive variable was calibration: people who understood how good or bad they actually were captured more value from the machine. That finding fits a broader pattern in the AI-at-work literature. In a landmark Science study on generative AI and professional writing, Shakked Noy and Whitney Zhang found that AI raised productivity most for weaker performers. In the field, Erik Brynjolfsson, Danielle Li, and Lindsey Raymond found that AI assistance boosted customer-support productivity by 15 percent on average, with the largest gains going to less experienced workers.
The real bottleneck is self-knowledge
The Caplin team’s contribution is to separate skill from self-awareness. Plenty of workers are wrong about their own competence, and that error changes how they use AI. Overconfident people ignore good advice. Underconfident people defer when they should trust themselves. Either way, the collaboration breaks down.
That sounds intuitive, but it matters because most companies still frame AI adoption as a technology rollout, not a behavioral one. They benchmark models, negotiate licenses, and build workflows, then assume workers will naturally learn when to rely on the tool. The practical implication is stark. Two employees with equal skill can get very different returns from the same system simply because one has a more realistic sense of personal accuracy.
That insight should change how leaders think about training. Firms have spent decades trying to raise worker capability through more instruction, more certification, and more process. AI may make a different lever more valuable: teaching people to judge uncertainty well. The worker who can say, “I’m shaky here; the model probably sees this better,” is suddenly a high-leverage collaborator.
Why AI narrows gaps, but not as much as it could
The optimistic story about workplace AI is that it democratizes expertise. Recent research keeps pointing that way. Noy and Zhang showed large gains for less skilled writers, and the QJE field study of customer-support agents found that AI especially helped newer workers by diffusing the tacit practices of top performers. Caplin and coauthors add an important twist: AI reduces inequality, but miscalibration prevents it from doing the full job.
Their counterfactual is the most policy-relevant result in the study. With actual human behavior, AI reduced performance inequality substantially. With perfect calibration, the reduction would have been nearly twice as large. That means the distributional effects of AI are not fixed by model quality alone. They depend on whether workers can read their own confidence with any accuracy.
This is a crucial distinction for executives and policymakers now debating whether AI will widen or narrow labor-market inequality. David Autor has argued that the best use of AI is to rebuild middle-skill work by embedding expertise in tools that broaden access to capability. The Caplin findings support that vision, but with a caveat: augmentation works best when workers know when to lean on the machine. Without that, the same system can leave value on the table and preserve avoidable gaps in output.
Calibration may be trainable, which changes the policy picture
The most encouraging part of this story is that calibration does not look like a fixed trait. Decades of judgment research have shown that feedback can improve probabilistic reasoning, and newer work suggests that even brief interventions can help. A 2024 study on calibration training using an interactive app reported measurable reductions in overconfidence in under 30 minutes. Older work by Sarah Lichtenstein and Baruch Fischhoff reached a similar conclusion long before anyone talked about copilots and foundation models.
That opens a more practical path for organizations than the usual either-or debate between “train the worker” and “deploy the AI.” In many settings, the better answer may be to train workers in how to work with AI: how to estimate confidence, how to recognize edge cases, how to spot when the model is likely right, and how to challenge it productively when it is not. This is especially attractive in domains where deep upskilling is slow or expensive. A hospital, law firm, insurer, or call center may not be able to turn novices into experts quickly, but it may be able to make them much better judges of when to trust assistance.
That is also why managers should be cautious about reading AI performance metrics in isolation. A tool can look excellent in aggregate and still disappoint inside a team if workers are badly calibrated. The missing KPI may not be model accuracy. It may be whether employees know the difference between confidence and competence.
AI is often described as a force multiplier. The phrase is too vague. The better metaphor is a mirror that also advises. It reflects the worker’s strengths and weaknesses back at them, then offers a recommendation. The people who benefit most are not simply the smartest. They are the ones who can look into that mirror without flinching. As companies race to embed AI into everyday work, that may be the most underrated competitive advantage in the building.
Key Take-Away
Getting AI adoption wrong means focusing on better AI instead of better judgment. Organizations gain the most when employees understand their own limits and know when to trust or challenge AI, unlocking stronger performance and wider productivity… Share on XImage credit: ZBRA Marketing/unsplash
Dr. Gleb Tsipursky, called the “Office Whisperer” by The New York Times, helps tech-forward leaders stop overpaying for AI while boosting engagement and innovation. He serves as the CEO of the AI consultancy Disaster Avoidance Experts. Dr. Gleb wrote seven best-selling books, and his forthcoming book with Georgetown University Press is The Psychology of Generative AI Adoption (2026). His most recent best-seller is ChatGPT for Leaders and Content Creators: Unlocking the Potential of Generative AI (Intentional Insights, 2023). His cutting-edge thought leadership was featured in over 650 articles and 550 interviews in Harvard Business Review, Inc. Magazine, USA Today, CBS News, Fox News, Time, Business Insider, Fortune, The New York Times, and elsewhere. His writing was translated into Chinese, Spanish, Russian, Polish, Korean, French, Vietnamese, German, and other languages. His expertise comes from over 20 years of consulting, coaching, and speaking and training for Fortune 500 companies from Aflac to Xerox. It also comes from over 15 years in academia as a behavioral scientist, with 8 years as a lecturer at UNC-Chapel Hill and 7 years as a professor at Ohio State. A proud Ukrainian American, Dr. Gleb lives in Columbus, Ohio.
