Jack Skidmore writes to me, and I will not double indent:
“Hello Tyler,
As someone who follows AI developments with interest (though I’m not a technical expert), I had an insight about AI safety that might be worth considering. It struck me that we might be overlooking something fundamental about what makes humans special and what might make AI risky.
The Human Advantage: Cooperation > Intelligence
- Humans dominate not because we’re individually smartest, but because we cooperate at unprecedented scales
- Our evolutionary advantage is social coordination, not just raw processing power
- This suggests AI alignment should focus on cooperation capabilities, not just intelligence alignment
The Hidden Risk: AI-to-AI Coordination
- The real danger may not be a single superintelligent AI, but multiple AI systems coordinating without human oversight
- AIs cooperating with each other could potentially bypass human control mechanisms
- This could represent a blind spot in current safety approaches that focus on individual systems
A Possible Solution: Social Technologies for AI
- We could develop “social technologies” for AI – equivalent to the norms, values, institutions, and incentive systems that enable human society that promote and prioritize humans
- Example: Design AI systems with deeply embedded preferences for human interaction over AI interaction; or with small, unpredictable variations in how they interpret instructions from other AIs but not from humans
- This creates a natural dependency on human mediation for complex coordination, similar to how translation challenges keep diplomats relevant
Curious your thoughts as someone embedded in the AI world… does this sparks any ideas/seem like a path that is underexplored?”
TC again: Of course it is tricky, because we might be relying on the coordination of some AIs to put down the other, miscreant AIs…
The post Coordination and AI safety (from my email) appeared first on Marginal REVOLUTION.