The Fourth Conference on Artificial General Intelligence

The End of Rough Equality of Intelligence

Bill Hibbard

Abstract: Natural humans all have roughly the same intelligence: we speak the same languages (English speakers can learn French, etc) and share the same daily skills. But with AI and human enhancement will come a divergence of intelligence levels of humans, or of the intelligence levels of machines serving different humans. If a person’s intelligence depends on how large a brain they can afford and their wealth depends on their intelligence, this positive feedback loop will create a divergence of intelligence levels. The most intelligent will speak languages that the less intelligent can never learn, which will freeze most out of any role in social decisions. It is an historical observation of human nature that power corrupts and absolute power corrupts absolutely.  The greatest AI threat will be the teaming of the power of superior intelligence with corruptable human nature. There will be political will for AI designs that prevent AI from taking control over all humans and prevent AI from wireheading, but politics are likely to work in favor of teaming AI with wealthy and powerful humans.