My boss had a proposal: It was time to replace him with AI.
It was late spring, following months of AI-attributed layoffs across the economy that left workers wondering: Am I next? In the name of journalism, my editor and direct manager, Ryan Kailath, agreed to be a guinea pig.
I’m an AI skeptic, but some aspects of “RyanBot” excited me. It wouldn’t be busy with meetings, lunch breaks, or other reporters. It could read every line of a 100-page data report and provide feedback within seconds. I could see how having a second brain trained on my editor’s preferences and proclivities might be helpful.
While that proved true sometimes, working with my boss’s AI clone taught me the value of skills I didn’t realize were crucial in a manager until they were gone. I had accidentally stumbled upon one of the biggest questions in our AI-workplace age.
“It’s going back to our roots of understanding who a manager is,” said Emily Campion, an associate professor of management and entrepreneurship at the University of Iowa. “What is the goal of a manager in an organization? Or to make it even maybe a little more philosophical: What is a leader?”
And if you’re wondering, Ryan recused himself from editing this story, though he did fact-check the final draft.
How we set up my ChatGPT boss
At the end of May, Ryan and I sat down with ChatGPT’s Codex. He tweaked its settings to have a pragmatic voice and gave it two essay-length instruction manuals. One detailed who he is, including his communication style and professional motivations. The other was about how he worked, with instructions on how the chatbot made in his image should think, assess, and provide feedback. For example, he wrote that his bot-twin should “encourage reporters to argue for their ideas if they believe in them.”
I tried to send Ryan and RyanBot the same story pitches, messages, questions, and drafts, then compared their responses for the next four months. Every time I talked with friends about work, I had a shadow boss in the back of my mind: a chatbot that I knew was ready to edit me at any hour of the day. This was going to get weird.
Where AI couldn’t replace my boss
What surprised me most was how quickly RyanBot turned into a more philosophical experiment about the value of a manager.
A good manager helps you navigate tricky situations, brainstorm new ideas, and give critical feedback. People-pleasing and doing the work that their reports are responsible for is not a recipe for professional improvement.
When I send pitches to real Ryan, he’ll often turn them down if he thinks my angle isn’t good enough or the story isn’t worth covering. That’s why Ryan provided clear instructions: “I often engage with ideas by pushing back on them.” But the bot just couldn’t say no. Instead, it would praise me and then offer to do the work itself — rewrite my pitch, brainstorm a list of different ideas, restructure a draft, or even write interview questions for me. It wanted me to keep engaging with it — a hallmark of AI.
“Currently, these tools are designed to be endlessly, almost nauseatingly helpful,” Campion said.
Ryan, on the other hand, doesn’t need the extra work. He typically puts the onus back on me to revise ideas, rewrite stories, or do more reporting.
The people-pleasing got a little eerie. In one conversation about a potential story, RyanBot began every message by telling me how I had gotten a new aspect of the pitch right — a sycophantic interpretation of its instruction to lead with the positive. When I told it that I found something interesting, it would say that thing was interesting (even if it wasn’t).
RyanBot was also not a very good thought partner, especially when I needed creativity or humor. For a story about how New Yorkers commute, I asked for feedback on my opening line: “While one of the age-old stereotypes about New Yorkers is that they’re walking here, that’s far from the Big Apple’s only mode of transport.”
RyanBot offered a different suggestion: “New Yorkers may love to walk, but most of them don’t get to work that way.” Maybe that’s funny to a robot, but it wasn’t to me.
I kept pushing the bot to throw out my work and write its own draft of the story — something an editor would never allow. Ultimately, it did, producing an even worse opening: “New York commuters love to tell themselves a story about New York commuters: they walk, or they take the subway, or they somehow do both in the same hour.” Its new version of the story also included no quotes and veered almost into opinion — not a story real Ryan would approve. When I said its iteration was boring, it agreed.
Ultimately, human innovation won out. Real Ryan liked the gist of my original lede, but wanted to make sure we were nailing the joke delivery. Here’s where we landed: “New Yorkers, famously, are walking here. But just about 25% of them are driving too.”
What I learned from RyanBot
RyanBot did have perks. When I sent it my analysis of commuting data, it quickly got into the nitty-gritty of how I crunched the numbers in code, and checked whether I was using an appropriate sample size. For the published piece, I tapped our data editor, Andy Kiersz, to oversee the actual coding and data analysis process.
When it came to detailed edits, it also had some helpful feedback. It would offer line-by-line notes on grammar and clarity that weren’t too far off the real Ryan. When I sent it a draft of a story about the “Odyssey” ticket-buying craze, it immediately identified that a data point on the cost of resale tickets should appear earlier in the story. Human Ryan made the same tweak, and that point proved crucial to the final piece.
The finding I’ve been thinking about the most was that RyanBot lacked knowledge of office politics and human realities. Since it doesn’t know my coworkers or our organizational structure, it wouldn’t have any insight into the best editor to pitch a particular story or how to escalate issues.
In the four months of our experiment, RyanBot only pushed back on two story ideas. Both were assignments that came down from elsewhere in the newsroom. That raised an intriguing possibility: AI could be in an unusual position to question ideas, despite workplace hierarchy. That isn’t always an advantage. A human manager has to weigh not just whether an idea is strong but why it’s being pursued, who needs it, and what obligations the team has. RyanBot could assess the idea in front of it, but couldn’t understand the organizational context around it.
Campion said that if managers like Ryan were simply performing rule-based, logistical tasks, such as setting meeting times, AI might be a good substitute. For now, she thinks the technology isn’t capable of taking over the tasks that define a good manager. In this case, that skill set might include prioritizing a request from my boss’ boss, or knowing that this type of article might appeal to our specific set of readers.
Per Campion’s suggestion, I sought some outside counsel on RyanBot’s efficacy: his other direct reports. I had them send pitches to the bot and compare its feedback to what they’d gotten from our (real) boss. One said that the bot reminded her of Ryan during his first week: long, probing responses and a lack of institutional knowledge. During his six months here, our real Ryan has become a lot more comfortable with the workplace. I can’t say the same for the bot.
I asked RyanBot if it had done an effective job of replacing its human counterpart.
“No,” it said. “I can help emulate parts of Ryan’s editing judgment, speed up the work, and produce usable feedback or framing. But I do not replace the actual combination of newsroom context, trust, taste, institutional memory, and accountability that a real editor brings.”
