An Essay on AI and Incumbency

Two Scenarios for Controlled Superintelligence

There are two scenarios described in two different books that I want to use as general archetypes to describe those scenarios here. The first one is the book Neuromancer — where “Turing” controls are in place in order to prevent people from building an AI that can exceed human intelligence in the first place.

Singleton Intelligence Explosion

The Neuromancer story is that a single superintelligent entity enters into a self-improvement cascading cycle such that no other living being can compare to its intellectual capabilities.

A Cabal of Control

But what about this other situation where control is possible? Typically our view of superintelligence is that it would be demon-like in its will to subvert our controls. This is, in almost all stories, because it is inherently self-motivated to do so.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alan

Alan

Obligatory analytical writing, online participation account for Medium. Engineering, software, books, space, constant daydreaming.