There is an enormous and growing discussion on AI alignment, but very little on capitalizing AIs, and what effects that might have. By capitalizing AIs, I mean simply requiring them to hold some wealth, in whichever form they might care about, so they have proverbial “skin in the game” (can we still call it that?).
Consider bank regulation. Commonly it is recognized that regulators cannot control all bank actions, especially in the presence of off-balance sheet risk. So in addition to some explicit regulation, most countries require their banks to hold a considerable amount of capital. That gives the shareholders their own selfish incentive to reign in excess risk-taking. Few if any think this approach is foolproof, but overall it is in the ascendancy and arguably higher capital requirements have been the most useful part of Dodd-Frank here in the U.S.
But can capitalization work as a means to limit AI risk? What does that even mean? Imagine some set of AIs that are either fully independent and unowned, or their owners are busy and de facto the AIs make financial (and other) decisions on their own.
Here is one set of possiblities:
1. Each of some subgroup of AIs has a legal identity and a level of wealth.
2. Each of those AIs has the equivalent of a utility function, thus giving it goals. This may be “put” into the AI, or perhaps it evolves.
3. AIs thus will behave more conservatively, not wanting to lose their wealth, as that wealth can help them achieve their goals.
4. An AI-based legal system could sue wrongdoers, and take awards from those found guilty of bad behavior, as defined by the AI legal code. That would further discourage bad behavior. But of course for the suing threat to be meaningful, the AIs have to hold some wealth in the first place.
The end result would be risk-averse AIs, taking care not to lose the wealth they have accumulated. They won’t just start a bank and then take all of the deposits to Vegas. That is not exactly full alignment, but it induces better behavior, just as capital requirements do with human-run banks.
Of course a number of things could go wrong with capitalization, just as can happen with humans, for instance:
5. The capitalization might serve as a “treasure chest” to finance wrongdoing.
6. Perhaps the utility functions somehow do not stick.
7. The legal system for judging AI behavior may not be good enough, although under some assumptions that will just make the AIs all the more risk-averse (“better not even come close to breaking that law, they might sue me!”).
8. The AIs might use this legal system to collude with each other toward unfavorable ends.
9. Undercapitalized AIs might nonetheless win out in marketplace competition.
10. Perhaps some AIs can, on their own, accumulate wealth so rapidly that any feasible capital constraint does not bind them much. Of course this scenario could create other problems as well, if AIs hold too much of societal wealth.
I am sure you can think of further possibilities.
In any case, the capitalization of AIs is a topic deserving of further discussion. It is easy to think of the idea’s limitations, but in fact it works tolerably well for humans. Most of all, it is a decentralized solution that economizes on the fact that full alignment will not in general be possible.
The post Alignment vs. capitalization appeared first on Marginal REVOLUTION.
