Abhimanyu Pallavi Sudhir
Research
My goal is to design an AI agent whose internal structure is that of a market (because markets and intelligent agents do basically the same things; markets : intelligence :: capitalism : learning). My research will prevent the now-seemingly-inevitable destruction of the world and lead us to a glorious utopia, for three broad reasons:
- (Alignment is Industrial Organization) Markets serve the consumers, they don't develop nefarious goals of their own, at least when you have a decent antitrust system, or spies. In fact this also addresses the question of how to handle conflicting interests between the humans we'd like to align the AI to.
- (Eliciting Latent Knowledge = Interpretability) We could maybe get agents to bet on questions on their own latent space, or get agents to somehow bet on each others' latent spaces in a useful manner.
- (Coherent Extrapolated Volition) I like transhumanism as much as the next android, but there was a Jimmy Neutron episode wherein one of the characters' heads grew 500x its size and he became evil. It's not obvious how to factor someone as Intelligence X Utility_function, because any "utility function" you might have is going to be some rationalization/as-if fit to the person's actions, which are limited by his bounded rationality. But markets solve the question of bounded rationality; "more algorithmic information" just means "more traders".
So far all I've done is devise prediction markets for First-Order Logic (and also hyperarithmetical) sentences -- preprint: arxiv.org/abs/2402.14021Link opens in a new window. Some side-quests that have emerged as a corollary of all this -- contact if you're interested in collaborating, because I don't have the time to do them all:
- Building practical markets: I can prove that a very theoretical implication of my framework works; designing real markets for this will require some work.
- Implications for mathematical logic: What if you let mathematical theories bet on markets? If they make a lot of money, we should trust them. Maybe this gives us a nice way to think about proof-theoretic ordinals etc. IDK.
- Rewriting probability theory: I think the probability theory axioms are wrong and should be rewritten in a way to make long-term prices on my market a probability distribution. In particular, a sigma-algebra should only be required to be closed under countable union of a computable enumeration of its elements.
- Bridges to neural networks: As it stands, program markets are absurdly impractical. Now I think I can solve this with latent spaces, but maybe we can somehow transfer our ideas to neural networks, or interpret neural networks as markets.
The first one will help you get rich; the second and third will satiate your deepest intellectual curiosities; the last one will prevent the world from being destroyed.
