
Robin Hanson at Singularity Summit 2009 -- How Does Society Identify Experts and When Does It Work?
2 years ago
Robin Hanson, associate professor of economics at George Mason University, talks about the kinds of knowledge that seem relevant to understanding "singularity," the social processes that have labeled folks as "experts" on this topic, and the degree of match between the two.
-
Vimeo: About / Blog / Developers / Jobs /
Community Guidelines /
Help Center / Video School / Music Store / Site Map
/ Vimeo
or
-
Legal: TM + ©2012 Vimeo, LLC. All rights reserved. / Terms of Service / Privacy Statement / Copyright

Prev week
To use Robin's excellent "rising tide" image, what happens when self-improving AIs become better than humans at everything? And by everything I mean EVERYTHING? That is, including designing new technology, creating beautiful art and engaging stories, determining what humans want, designing systems of ethics and law, and even convincing people that they are human (so that even those who are willing to pay for "authentic human interaction" will be better served by robots)? If you are thinking, "but humans will still.." I'll cut you off right there and ask why a self-improving AI wouldn't replace us there as well.
In such a scenario, I can only see two potential advantages for humans. One is that we could also recursively self-improve and keep up with machines that way...though that gets into rather sticky philosophical issues of the dividing line between human and machine blurring into meaninglessness--not a problem in and of itself, but what if it leads to us abandoning everything we value (nickbostrom.com/fut/evolution.html)?
The second potential advantage I can see is that humans will own the machines and thus can set their utility functions, tell them what to do, and so on. But even then, it seems likely to me that only a small fraction of the human population will own the means of production for these machines and thus produce everything and thus possess all of the wealth...so what happens to everyone else?