Arvind Narayanan, a Princeton computer science professor and co-author of AI Snake Oil, discusses the gap between AI hype and reality, focusing on reasoning models, AI agents, evaluation challenges, policy implications, and the future of education and work.
Reasoning Models and Their Uneven Progress
Recent reasoning models (like OpenAI’s o1/o3) show impressive results in domains with clear correct answers: math, coding, and certain scientific tasks.
The big open question is how far this performance generalizes beyond narrow, easily-verified domains.
Historical parallel: 10 years ago, reinforcement learning excelled at games like Atari but failed to generalize far beyond them. Reasoning models could follow the same path—or they could extend reasoning to domains like law and medicine by combining code/internet access with logical inference.
Construct validity problem: Benchmarks like SWE-bench (real GitHub issues, created by Narayanan’s Princeton colleagues) are better than toy problems, but still far from the messy reality of actual software engineering. Dramatic benchmark improvements don’t necessarily translate to dramatic productivity gains.
Real-world evaluations are needed: domain-specific tests, randomized controlled “uplift” studies, and simply observing how people actually use the tools (“vibes”).
Inference Scaling and Verifier Imperfections
Inference scaling (test-time compute) is a major area of investment: instead of just making models bigger at training time, you spend more compute at inference time to improve answers.
One approach pairs a generative model with a verifier (unit tests for code, theorem checkers for math). The hope is that verifiers are perfect and deterministic, so the model can generate millions of solutions until one passes.
Narayanan’s paper “Inference Scaling Flaws” shows that when the verifier is even slightly imperfect, inference scaling saturates quickly—sometimes within ~10 invocations rather than the hoped-for millions.
This has implications for scaling into domains without easy verifiers (law, medicine, etc.), where human review teams would be expensive and still imperfect.
Agentic AI: Tools vs. Autonomous Actions
“Agentic AI” is not one category—it’s useful to distinguish:
Generative agents (e.g., Google Deep Research): produce a report or output for a human expert to review. Low cost of errors, well-motivated as time-saving tools.
Autonomous action agents (e.g., booking flights, ordering food): take actions on the user’s behalf. High cost of errors (wrong flight, wrong address), and the real challenge is eliciting user preferences—which often takes 10-15 rounds of iteration even for humans.
Flight booking is almost the worst case for an AI agent: preferences are complex and often only discovered through seeing results, error rates of even 1% are intolerable, and early systems have already made costly mistakes (e.g., DoorDash delivering to the wrong address).
Narayanan is optimistic about agents gradually evolving (chatbots already search the web and run code), but skeptical of near-term autonomous action agents.
Evaluating AI Agents and Collaboration
Current state of agent evaluations resembles early chatbot benchmarks: static tasks (fixing GitHub issues, navigating simulated web environments) that don’t capture real-world complexity.
Key limitations:
Capability-reliability gap: A 90% score doesn’t tell you whether the agent is great at 9/10 tasks or fails 10% of the time at every task. For autonomous agents, the latter is catastrophic.
Safety: Safety-specific benchmarks exist, but safety should be part of every benchmark. Some web benchmarks require agents to take stateful actions on real websites, which could generate spam. Others use simulated environments that lose real-world nuance.
Control: Current agent frameworks (e.g., AutoGPT) can take unintended actions online. The only current safeguard is escalating every action to a human—essentially babysitting.
Middle ground: Use benchmarks as a necessary but not sufficient condition, then test top-performing agents in semi-realistic environments with humans in the loop.
Narayanan’s “AI Agent Zoo”: His team built a collaborative environment where multiple agents work together on tasks (e.g., writing a joke). Even simple tasks generated millions of tokens as agents explored their environment, understood collaborators, and produced output—suggesting overall inference costs will continue rising even as per-token costs fall.
Lessons from Past Technology Waves
Industrial Revolution / electricity: Took decades to figure out how to reorganize labor and factory layouts to take advantage of new technology. The shift from one big steam boiler to distributed electricity enabled the assembly line. We’re in similarly early stages with human-agent teams.
Internet: Transformed how we do almost every cognitive task, yet GDP impact has been minimal (“the computer revolution shows up everywhere except in the productivity statistics”). When old bottlenecks are removed, new ones emerge. Job categories remain largely the same as 20-30 years ago.
AI’s economic impact may follow the internet pattern: transformative in how we work, but with GDP impact unfolding over decades, not years.
Future of work: As cognitive tasks become automated, “work” may shift toward what we now call AI alignment and safety—supervising AI systems and making value-based judgments that we’re not comfortable delegating to machines.
Regulatory and Policy Implications
Export controls (e.g., the AI Diffusion Rule, chip/model export restrictions) have a historically mixed record of effectiveness. They’re more effective on hardware than on models, which are getting smaller and harder to contain. Inference scaling also means even existing open models can be powerful.
Diffusion vs. innovation: Political scientist Jeffrey Ding argues that the focus should be less on innovation and more on diffusion—how countries adopt, reorganize institutions, and adapt laws/norms to benefit from available technology. This is the real determinant of economic growth from technology.
US adoption: A recent paper claimed 40% of people use generative AI, but intensity is low (0.5-3 hours/week). Controlling for intensity, adoption may actually be slower than PC adoption. Reasons include the tools not yet being broadly useful, and confusion/hesitation (especially among students who see AI primarily as a cheating tool).
Education policy as low-hanging fruit: Teaching productive AI use (and pitfalls) at K-12 and college levels could significantly improve adoption. Narayanan allows AI in his classes but requires disclosure of how it’s used—a model for teaching AI literacy.
Flaws in Predictive AI and Lessons for Generative AI
Past failures in predictive AI (criminal justice risk scores, automated hiring) were not primarily technical failures—they reflected the fundamental difficulty of predicting future behavior. Social science evidence strongly supports this.
Generative AI won’t fix these problems if the underlying application is flawed. The limitations are about the application domain, not the technology.
Regulation should be expected in domains where AI makes consequential decisions. The question isn’t whether to regulate, but how to balance safety/rights with AI benefits.
Explainability in regulation doesn’t mean mechanistic interpretability—it means transparency about training data, audits, and expected behavior in new deployment settings.
Academia’s Role in AI Development
Academia has faced a compute crisis, making it harder to compete with industry on frontier model training. But opportunities remain:
New architectures and “blue sky” ideas that can be validated at small scale.
Innovations on top of existing models (e.g., new inference scaling methods).
Non-technical research: Interdisciplinary work on AI’s societal impacts, applications across domains, and serving as a counterweight to industry interests.
Narayanan argues that while 80% of computer science academia can align with industry, ~20% should explicitly provide independent scrutiny—similar to how medical researchers maintain distance from pharma.
AI in Scientific Research
AI for Science is a hot area with some overblown early claims (discovered “AI breakthroughs” that didn’t reproduce), but AI is already having real impacts:
Thinking partner: Using AI to critique ideas, do enhanced literature searches (semantic search across concepts, not just keywords—Narayanan uses it to search his own 100,000-word book).
Domain-specific tools: Various tools tailored to particular scientific fields.
Narayanan is excited about this area despite pushing back on extreme claims.
AI and Human Minds
Researchers are exploring the relationship between AI and human cognition from multiple angles:
Philosophers (e.g., Seth Lazar): Studying the ethical reasoning these models exhibit behaviorally (not ascribing morality, but comparing how models reason about ethics vs. humans).
Cognitive scientists (e.g., Tom Griffiths): Using AI both to learn from human minds for better AI design, and as a tool to better understand human cognition.
Future of Education
Narayanan is in the “not that much will change” camp regarding fundamental education:
Online courses (e.g., Coursera) were supposed to revolutionize education but didn’t, because the value of classroom education isn’t information transmission—it’s the social preconditions for learning: motivation, connections, caring, individualized feedback.
AI can personalize and motivate, but removing the human element loses something essential.
Inequality concern: AI’s impact on kids will have high variance. Wealthier families with time and resources to monitor and guide AI use will benefit enormously; others may face addiction and unhealthy use patterns (AI addiction can be highly personalized, like social media but worse).
Narayanan’s personal approach: He’s “tech-forward” with his kids, using apps like Khan Academy and building custom AI learning tools (e.g., a phonics app, a clock-face app for teaching time via Claude’s artifacts feature). He expects kids will use AI in bigger ways in the future, mostly outside schools rather than in them.
Quickfire Round
Overhyped: Agents (potential is real, but hype is out of control).
Underhyped: Boring but economically valuable applications (e.g., AI summarizing C-SPAN meetings for lawyers, translating old codebases like COBOL to modern languages).
Model progress in 2025: Depends on perspective—inference scaling may push specific verifiable tasks forward dramatically, but broader tasks (translation, etc.) may not improve as much.
Go-to model test: Play rock paper scissors, ask the model to go first, then ask how you won every time. Models consistently fail to understand turn-taking/context—revealing they lack awareness of their deployment context.
Agents by end of 2025: Many agentic workflows for generative tasks, but relatively few applications where AI autonomously does things for you.
AGI timeline: Narayanan prefers to ask about transformative economic impact (massive GDP impact) rather than AGI. His view: decades away, not years.
Weirdest prediction: Companies will train younger users to expect chatbots as the default way to access any information. Older generations will find this as strange as younger people find “going to the library.”
Most interesting startups: The “boring” ones (C-SPAN summaries, COBOL translation) and AI that disappears into everyday life (form factor discussion—glasses, ambient AI).
One policy change: Outlaw the term “AI” and be specific about the application. This would cut hype and bring clarity to discourse.
Most interesting non-academic role: A big tech company, to see the full picture of how people interact with AI at scale.
Future research directions: His lab is focused on agents (grounded evaluation, pushing back on hype, exploring potential). He’s also writing about “AI as normal technology”—arguing AI’s impact will unfold over decades like the internet, not revolutionize everything in 2-2 years.
Where to Learn More
Newsletter: AI Snake Oil (co-authored with Sayash Kapoor)—pushes back on hype while providing balanced coverage of AI’s positives and negatives.
