Before Emily Bender and I have looked at a menu, she has dismissed artificial intelligence chatbots as “plagiarism machines” and “synthetic text extruders”. Soon after the food arrives, the professor of linguistics adds that the vaunted large language models (LLMs) that underpin them are “born shitty”.
Since OpenAI launched its wildly popular ChatGPT chatbot in late 2022, AI companies have sucked in tens of billions of dollars in funding by promising scientific breakthroughs, material abundance and a new chapter in human civilisation. AI is already capable of doing entry-level jobs and will soon “discover new knowledge”, OpenAI chief Sam Altman told a conference this month.
According to Bender, we are being sold a lie: AI will not fulfil those promises, and nor will it kill us all, as others have warned. AI is, despite the hype, pretty bad at most tasks and even the best systems available today lack anything that could be called intelligence, she argues. Recent claims that models are developing a capacity to understand the world beyond the data they are trained on are nonsensical. We are “imagining a mind behind the text”, she says, but “the understanding is all on our end”.
Bender, 51, is an expert in how computers model human language. She spent her early academic career in Stanford and Berkeley, two Bay Area institutions that are the wellsprings of the modern AI revolution, and worked at YY Technologies, a natural language processing company. She witnessed the bursting of the dotcom bubble in 2000 first-hand.
Her mission now is to deflate AI, which she will only refer to in air quotes and says should really just be called automation. “If we want to get past this bubble, I think we need more people not falling for it, not believing it, and we need those people to be in positions of power,” she says.
In a recent book called The AI Con, she and her co-author, the sociologist Alex Hanna, take a sledgehammer to AI hype and raise the alarm about the technology’s more insidious effects. She is clear on her motivation. “I think what it comes down to is: nobody should have the power to impose their view on the world,” she says. Thanks to the huge sums invested, a tiny cabal of men has the ability to shape what happens to large swaths of society and, she adds, “it really gets my goat”.
Her thesis is that the whizzy chatbots and image-generation tools created by OpenAI and rivals Anthropic, Elon Musk’s xAI, Google and Meta are little more than “stochastic parrots”, a term that she coined in a 2021 paper. A stochastic parrot, she wrote, is a system “for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning”.
The paper shot her to prominence and triggered a backlash in AI circles. Two of her co-authors, senior members of the ethical AI team at Google, lost their jobs at the company shortly after publication. Bender has also faced criticism from other academics for what they regard as a heretical stance. “It feels like people are mad that I am undermining what they see as the sort of crowning achievement of our field,” she says.
The controversy highlighted tensions between those looking to commercialise AI fast and opponents warning of its harms and urging more responsible development. In the four years since, the former group has been ascendant.
We’re meeting in a low-key sushi restaurant in Fremont, Seattle, not far from the University of Washington where Bender teaches. We are almost the only patrons on a sun-drenched Monday afternoon in May, and the waiter has tired of asking us what we might like after 30 minutes and three attempts. Instead we turn to the iPad on the table, which promises to streamline the process.
It achieves the opposite. “I’m going to get one of those,” says Bender: “add to cart. Actual food may differ from image. Good, because the image is grey. This is great. Yeah. Show me the . . . where’s the otoro? There we go. Ah, it could be they don’t have it.” We give up. The waiter returns and confirms they do in fact have the otoro, a fatty cut of tuna belly. Realising I’m British, he lingers to ask which football team I support, offers his commiserations to me on Arsenal finishing as runners-up this season and tells me he is a Tottenham fan. I wonder if it’s too late to revert to the iPad.
Menu
Kamakura Japanese Cuisine and Sushi
3520 Fremont Ave N, Seattle, 98103
Otoro nigiri x2 $31.90
Salmon nigiri x2 $8
Agedashi x2 $8
Avocado maki $5.95
Edamame $3.50
Barley tea x2 $5
Total (including tax and tip) $82.56
Bender was not always destined to take the fight to the world’s biggest companies. A decade ago, “I was minding my own business doing grammar engineering,” she says. But after a wave of social movements, including Black Lives Matter, swept through campus, “I started asking, well, where do I sit? What power do I have and how can I use it?” She set up a class on ethics in language technology and a few years later found herself “having just unending arguments on Twitter about why language models don’t ‘understand’, with computer scientists who didn’t have the first bit of training in linguistics”.
Eventually, Altman himself came to spar. After Bender’s paper came out, he tweeted “i am a stochastic parrot, and so r u”. Ironically, given Bender’s critique of AI as a regurgitation machine, her phrase is now often attributed to him.
She sees her role as “being able to speak truth to power based on my academic expertise”. The truth from her perspective is that the machines are inherently far more limited than we have been led to believe.
Her critique of the technology is layered on a more human concern: that chatbots being lauded as a new paradigm in intelligence threaten to accelerate social isolation, environmental degradation and job loss. Training cutting-edge models costs billions of dollars and requires enormous amounts of power and water, as well as workers in the developing world willing to label distressing images or categorise text for a pittance. The ultimate effect of all this work and energy will be to create chatbots that displace those whose art, literature and knowledge are AI’s raw data today.
“We are not trying to change Sam Altman’s mind. We are trying to be part of the discourse that is changing other people’s minds about Sam Altman and his technology,” she says.
The table is now filled with dishes. The otoro nigiri is soft, tender and every bit as good as Bender promised. We have both ordered agedashi tofu, perfectly deep-fried so it remains firm in its pool of dashi and soy sauce. Salmon nigiri, avocado maki and tea also dot the space between us.
Bender and Hanna were writing The AI Con in late 2024, which they describe in the book as the peak of the AI boom. But since then the race to dominate the technology has only intensified. Leading companies including OpenAI, Anthropic and Chinese rival DeepSeek have launched what Google’s AI team describe as “thinking models, capable of reasoning through their thoughts before responding”.
The ability to reason would represent a significant milestone on the journey towards AI that could outperform experts across the full range of human intelligence, a goal often referred to as artificial general intelligence, or AGI. A number of the most prominent people in the field — including Altman, OpenAI’s former chief scientist and co-founder Ilya Sutskever and Elon Musk have claimed that goal is at hand.
Anthropic chief Dario Amodei describes AGI as “an imprecise term which has gathered a lot of sci-fi baggage and hype”. But by next year, he argues, we could have tools that are “smarter than a Nobel Prize winner across most relevant fields”, “can control existing physical tools” and “prove unsolved mathematical theorems”. In other words, with more data, computing power and research breakthroughs, today’s AI models or something that closely resembles them could extend the boundaries of human understanding and cognitive ability.
Bender dismisses the idea, describing the technology as “a fancy wrapper around some spreadsheets”. LLMs ingest reams of data and base their responses on the statistical probability of certain words occurring alongside others. Computing improvements, an abundance of online data and research breakthroughs have made that process far quicker, more sophisticated and more relevant. But there is no magic and no emergent mind, says Bender.
“If you’re going to learn the patterns of which words go together for a given language, if it’s not in the training data, it’s not going to be in the output system. That’s just fundamental,” she says.
In 2020, Bender wrote a paper comparing LLMs to a hyper-intelligent octopus eavesdropping on human conversation: it might pick up the statistical patterns but would have little hope of understanding meaning or intent, or of being able to refer to anything outside of what it had heard. She arrives at our lunch today sporting a pair of wooden octopus earrings.
There are other sceptics in the field, such as AI researcher Gary Marcus, who argue the transformational potential of today’s best models has been massively oversold and that AGI remains a pipe dream. A week after Bender and I meet, a group of researchers at Apple publish a paper echoing some of Bender’s critiques. The best “reasoning” models today “face a complete accuracy collapse beyond certain complexities”, the authors write — although researchers were quick to criticise the paper’s methodology and conclusions.
Sceptics tend to be drowned out by boosters with bigger profiles and deeper pockets. OpenAI is raising $40bn from investors led by SoftBank, the Japanese technology investor, while rivals xAI and Anthropic have also secured billions of dollars in the last year. OpenAI, Anthropic and xAI are collectively valued at close to $500bn today. Before ChatGPT was launched, OpenAI and Anthropic were valued at a fraction of that and xAI didn’t exist.
“It’s to their benefit to have everyone believe that it is a thinking entity that is very, very powerful instead of something that is, you know, a glorified Magic 8 Ball,” says Bender.
We have been talking for an hour and a half, the bowl of edamame beans between us steadily dwindling, and our cups of barley tea have been refilled more than once. As Bender returns to her main theme, I notice she has quietly constructed an origami bird from her chopstick wrapper. AI’s boosters might be hawking false promises, but their actions have real consequences, she says. “The more we build systems around this technology, the more we push workers out of sustainable careers and also cut off the entry-level positions . . . And then there’s all the environmental impact,” she says.
Bender is entertaining company, a Cassandra with a wry grin and twinkling eye. At times it feels she is playing up to the role of nemesis to the tech bosses who live down the Pacific coast in and around San Francisco.
But where Bender’s bêtes noires in Silicon Valley might gush over the potential of the technology, she can seem blinkered in another way. When I ask her if she sees one positive use for AI, all she will concede is that it might help her find a song.
I ask how she squares her twin claims that chatbots are bullshit generators and capable of devouring large portions of the labour market. Bender says they can be simultaneously “ineffective and detrimental”, and gives the example of a chatbot that could spin up plausible-looking news articles without any actual reporting — great for the host of a website making money from click-based advertising, less so for journalists and the truth-seeking public.
She argues forcefully that chatbots are born flawed because they are trained on data sets riddled with bias. Even something as narrow as a company’s policies might contain prejudices and errors, she says.
Aren’t these really critiques of society rather than technology? Bender counters that technology built on top of the mess of society doesn’t just replicate its mistakes but reinforces them, because users think “this is so big it is all-encompassing and it can see everything and so therefore it has this view from nowhere. I think it is always important to recognise that there is no view from nowhere.”
Bender dedicates The AI Con to her two sons, who are composers, and she is especially animated describing the deleterious impact of AI on the creative industries.
She is scathing, too, about AI’s potential to empathise or offer companionship. When a chatbot tells you that you are heard or that it understands, this is nothing but placebo. “When Mark Zuckerberg suggests that there’s a demand for friendships beyond what we actually have and he’s going to fill that demand with his AI friends, really that’s basically tech companies saying, ‘We are going to isolate you from each other and make sure that all of your connections are mediated through tech’.”
Yet employers are deploying the technology, and finding value in it. AI has accelerated the rate at which software engineers can write code, and more than 500mn people regularly use ChatGPT.
AI is also a cornerstone of national policy under US President Donald Trump, with superiority in the technology seen as being essential to winning a new cold war with China. That has added urgency to the race and drowned out calls for more stringent regulations. We discuss the parallels between the hype of today’s AI moment and the origins of the field in the 1950s, when mathematician John McCarthy and computer scientist Marvin Minsky organised a workshop at Dartmouth College to discuss “thinking machines”. In the background during that era was an existential competition with the Soviet Union. This time the Red Scare stems from fear that China will develop AGI before the US, and use its mastery of the technology to undermine its rival.
This is specious, says Bender, and beating China to some level of superintelligence is a pointless goal, given the country’s ability to catch up quickly, which was demonstrated by the launch of a ChatGPT rival by DeepSeek earlier this year. “If OpenAI builds AGI today, they’re building it for China in three months.”
Nonetheless, competition between the two powers has created huge commercial opportunities for US start-ups. On Trump’s first full day of his second term, he invited Altman to the White House to unveil Stargate, a $500bn data centre project designed to cement the US’s AI primacy. The project has since expanded abroad, in what those involved describe as “commercial diplomacy” designed to bolster America’s sphere of influence using the technology.
If Bender is right that AI is just automation in a shiny wrapper, this unprecedented outlay of financial and political capital will achieve little more than the erosion of already fragile professions, social institutions and the environment.
So why, I ask, are so many people convinced this is a more consequential technology than the internet? Some have a commercial incentive to believe, others are more honest but no less deluded, she says. “The emperor has no clothes. But it is surprising how many people want to be the naked emperor.”
George Hammond is the FT’s venture capital correspondent
Find out about our latest stories first — follow FT Weekend on Instagram, Bluesky and X, and sign up to receive the FT Weekend newsletter every Saturday morning
