The rapidly evolving technology has already started to disrupt day-to-day activities in professional settings, and leaders at the forefront of the AI revolution have been clear about how they hope to implement systems like ChatGPT and Gemini into the mainstream workflow.
But while many employees may be cautiously skeptical about the impending AI overhaul, a recent study found that people are actually mistrustful of artificial intelligence for the wrong reasons while frequently trusting in the technology for tasks it’s more likely to mess up.
The September 2023 study, which is titled “How People Can Create—and Destroy—Value with Generative AI,” was spearheaded by François Candelon, the managing director and senior partner at consulting company Boston Consulting Group.
The study’s findings are back in the news this week after Candelon sat down with the Wall Street Journal’s Executive Insights podcast to discuss generative AI in the workplace.
Candelon partnered with talent from top universities like MIT, Wharton, Harvard Business School, and the University of Warwick, and used his consulting company’s own employees to execute the experiment, which he told The Journal was inspired by his desire to figure out how humans and AI can work together to help businesses.
The more than 750 study participants were given real tasks, including “creative product innovation” assignments. The participants were instructed to use OpenAI tool GPT-4 to help them with tasks like pitching the shoe concepts to their boss, coming up with focus group questions, and executing a successful social media rollout, Candelon said.
The study found that people using AI faired much better than those working without it when it came to creative product innovation tasks. About 90% of the participants improved their performance when using AI for any task involving ideation and content creation.
Participants also converged on a performance level that was 40% higher than those working on the same task without GPT-4, according to the study.
The most benefits were seen when people didn’t try to change or improve the technology’s output suggestions, accepting GPT-4’s suggestions as is, the study found.
But there are still some tasks where humans have the edge. People’s problem-solving skills far outweigh the help offered by AI, Candelon said.
The study found that generative AI actually persuaded several participants to accept GPT’s misleading output, even when they had been briefed on the possibility of wrong answers.
Participants who used AI for problem-solving tasks performed 23% worse than those who didn’t use the tool at all, according to the study.
The “double-edged sword” that is generative AI, with its “relatively uniform output,” can also reduce a group’s diversity of thought by 41%, the study found.
But Candelon stressed to The Journal that AI is exceedingly powerful and, ultimately, unavoidable.
“There is this famous quote saying that humans won’t get replaced by AI. They will get replaced by humans using AI,” he told the outlet.
Candelon said the study shows that data will become even more important with generative AI in the workplace, forcing people to revisit their workflows and figure out places for human and AI collaboration.
