OpenAI’s president, Greg Brockman, said deciding which teams get graphic processing units inside the company is an exercise in “pain and suffering.”
Brockman said on an episode of the “Matthew Berman” podcast published Thursday that managing the crucial resource is emotional and exhausting.
“It’s so hard because you see all these amazing things, and someone comes and pitches another amazing thing, and you’re like, yes, that is amazing,” he said.
He explained that the company divides its computing power between research and applied products. The company’s chief scientist and research head decide allocations within the research side. Senior leadership — CEO Sam Altman and the CEO of applications, Fidji Simo — decide the overall split between research and applied teams.
At the operational level, a small internal team focuses on shuffling GPU assignments, including Kevin Park, who is responsible for redistributing hardware as projects wind down.
“You go to him and you’re just like, ‘OK, like we need this many more GPUs for this project that just came up,'” Brockman said. “And he’s like, ‘All right, there’s like these five projects that are sort of winding down,'” he added.
The internal GPU shuffle reflects the broader scarcity that OpenAI has warned about for months. Brockman said compute drives the productivity of entire teams — and the stakes are high.
“People really care,” he said. “The energy and emotion around, ‘Do I get my compute or not?’ is something you cannot understate.”
Brockman and OpenAI did not respond to a request for comment from Business Insider.
The race for GPUs
OpenAI has been vocal about its insatiable demand for computing power.
Related stories
Business Insider tells the innovative stories you want to know
Business Insider tells the innovative stories you want to know
“Every time we get more GPUs, they immediately get used,” OpenAI’s chief product officer, Kevin Weil, said on an episode of the “Moonshot” podcast published in August.
Weil said the need for compute is simple: “The more GPUs we get, the more AI we’ll all use.” He highlighted that adding bandwidth made the explosion of video possible.
Altman said last month that OpenAI is launching “new compute-intensive offerings.” Because of the costs involved, some features will initially be limited to Pro subscribers, while certain new products will have extra fees, he added.
Altman framed the push as an experiment in stretching AI infrastructure to its limits: “We also want to learn what’s possible when we throw a lot of compute, at today’s model costs, at interesting new ideas,” he wrote on X.
Other tech giants have also been blunt about their appetite for GPUs.
Mark Zuckerberg said on an episode of the “Access” podcast published last month that Meta is making “compute per researcher” a competitive advantage. He said the company is outspending rivals on GPUs and the custom infrastructure needed to power them.
