Startup Drama Exposes Legal Headache of AI Agent Era

There’s AI agent drama in startup land, and it poses a big legal question.

It started with an accusation from Josh Manders, founder of the cloud hosting platform Primcloud. Manders asked why Jake Runzer, an employee of Railway, a competing cloud provider, was “trying to breach” Primcloud’s infrastructure.

Railway CEO Jake Cooper responded and tagged Runzer.

“That was indeed me,” Runzer wrote, saying that no malice was intended. “I’ve been running some agents to do signups + agentic testing on similar platforms. Just halted the harness I was designing around this as it was definitely going off the rails and reaching too far.”

Runzer’s post quickly circulated across the web as X users questioned the legality of AI agents attempting to breach a company’s infrastructure. Cooper apologized and said that Railway would ban any testing of the models on external organizations that wasn’t performed manually.

The legal threat of the AI era

Runzer said that his agent went “off the rails.” The drama drives at a key question of the AI age: If an agent gets in trouble, who could be held legally responsible?

It’s a question that has challenged lawyers, especially after the OpenAI-Hugging Face hack. Since AI agents can operate autonomously, proving liability can be complicated.

Mark Rasch, the general counsel of threat advisory firm Unit 221B, said that prosecutors would have to prove two things: that the AI agent actually attempted to illegally access another computer, and that the user intended for it to do so.

That second part — proving the user’s intent to illegally breach a system — is thorny. In the case of a bug bounty, or a deal offered from a company to security researchers who can find flaws in its system, users look for security vulnerabilities but have no intent to break the law. “The problem is that the AI agent, once they find a way to break in, might say: ‘Huh, this might work, let me try it,'” Rasch said.

Then there’s the question of negligence. Elimu Kajunju, a cybersecurity lawyer at Rimon Law, said defendants could try to shift blame onto the AI companies that built the agents.

Kajunju thought these cases were becoming harder to defend, because it’s difficult to claim surprise. “This has happened multiple times now,” he said, speaking generally. “We really can’t say that anymore.”

Both Rasch and Kajunju said that the legal precedent was still forming, making it a pressing question for lawyers and AI users alike.

Where the spat stands

Railway’s CEO later posted a longer explanation of the incident.

Cooper said that an employee, on his own time, had created a tool for “proactive identification + reporting of bug bounties within the broader community,” using an externally hosted service running on DeepSeek. The employee’s tool was used only once, the CEO said, and the employee “halted it proactively within 15 minutes of starting.”

“Railway does not condone these actions,” Cooper wrote. “We create an environment where internal red teaming has strict guidelines, and when bug bounty disclosures must occur, we follow every letter of responsible disclosure. We have addressed this incident with the employee directly and have taken appropriate action. We are also reinforcing our expectations for employee conduct to help prevent similar situations.”

Manders countered that he doesn’t have a bug bounty program. Primcloud considers any attempt to “probe, scan or test” its vulnerabilities as a security violation in its acceptable use policy.

To further complicate Railway’s week, the company sustained multi-day outages starting Wednesday. By Friday, it said that some customers’ service had been restored.

It’s not clear whether Manders will sue. Manders, Runzer, and Cooper did not respond to requests for comment from Business Insider.

Cooper had some thoughts on who was to blame.

“You operate the machinery, you are responsible for that what that machinery does,” he wrote.