If your personal AI assistant sends you what looks like intimate details about someone else’s life, chances are it made up the whole thing.
At least, that’s what Instinct’s creator, Noah Shinn, said happened when a user of the viral AI tool said an agent admitted to leaking someone else’s conversation into their chat.
The poster, @prit4k, uploaded screenshots of their interaction with Instinct on Tuesday, which show that the assistant seemingly gave them someone else’s middle name and discussed a photo they never sent of a document mentioning money.
Then, Instinct apologized and straight up told @prit4k that the photo “got crossed into” their conversation, calling it a “serious glitch.”
“Was this a data leak or a hallucination?” @prit4k wrote in alarm on X.
Shinn responded that this instance was just a hallucination. The agent had made up a proper noun and “further amplified” the mistake with its reasoning, he said.
“This was not a data breach, no user data was shared, and no user isolation boundary was violated,” Shinn wrote in a post on Wednesday evening.
The 23-year-old did not say how he knew that @prit4k’s case was a hallucination, but touted Instinct’s privacy measures.
“We’ve done a lot of work to make sure that users’ data are truly partitioned, from isolated sandboxes to short-lived local credentials to identity-signed tool execution,” he wrote.
Shinn did not immediately respond to a request for comment sent outside regular business hours by Business Insider.
He said in his post that his team had worked over the last 48 hours on an “active hallucination detection system” that can predict and detect such errors.
“This layer has the ability to steer or intercept Instinct from proceeding with the next thinking trace or tool call execution before it is generated or executed,” he wrote, saying he would share more about the system in the coming weeks.
Instinct, an invite-only AI assistant, took Silicon Valley by storm this summer, enabling Shinn’s startup, Spear Street, to raise $250 million at a $2.5 billion valuation later. Business Insider’s Pranav Dixit tried the assistant and wrote earlier this month that he was impressed by how it intuitively helped him with everyday tasks.
AI assistants like Instinct, which help users run everyday tasks like calling insurance companies or booking movie tickets, are turning into all the rage in the tech world. Meta released its own agent, Muse, on September 8, and it’s already hit the top spot among the most-downloaded apps for Apple and Android devices.
