The unfinished fable of the sparrows -- Preface -- Past developments and present capabilities : Growth modes and big history ; Great expectations ; Seasons of hope and despair ; State of the art ; Opinions about the future of machine intelligence -- Paths to superintelligence : Artificial intelligence ; Whole brain emulation ; Biological cognition ; Brain-computer interfaces ; Networks and organizations -- Forms of superintelligence : Speed superintelligence ; Collective superintelligence ; Quality superintelligence ; Direct and indirect reach ; Sources of advantage for digital intelligence -- The kinetics of an intelligence explosion : Timing and speed of the takeoff ; Recalcitrance : Non-machine intelligence paths; Emulation and AI paths ; Optimization power and explosivity -- Decisive strategic advantage : Will the frontrunner get a decisive strategic advantage? ; How large will the successful project be? : Monitoring; International collaboration ; From decisive strategic advantage to singleton -- Cognitive superpowers : Functionalities and superpowers ; An AI takeover scenario ; Power over nature and agents -- The superintelligent will : The relation between intelligence and motivation ; Instrumental convergence : Self-preservation; Goal-content integrity; Cognitive enhancement; Technological perfection; Resource acquisition -- Is the default outcome doom? : Existential catastrophe as the default outcome of an intelligence explosion? ; The treacherous turn ; Malignant failure modes : Perverse instantiation; Infrastructure profusion; Mind crime --
The control problem : Two agency problems ; Capability control methods : Boxing methods; Incentive methods; Stunting; Tripwires ; Motivation selection methods : Direct specification; Domesticity; Indirect normativity; Augmentation
Oracles, genies, sovereigns, tools : Tool AIs ; Comparison
Multipolar scenarios : Of horses and men : Wages and unemployment; Capital and welfare; The Malthusian principle in a historical perspective; Population growth and investment ; Life in an algorithmic economy : Voluntary slavery, casual death; Would maximally efficient work be fun?; Unconscious outsourcers?; Evolution is not necessarily up ; Post-transition formation of a singleton? : A second transition; Superorganisms and scale economies; Unification by treaty
Acquiring values : The value-loading problem ; Evolutionary selection ; Reinforcement learning ; Associative value accretion ; Motivational scaffolding ; Value learning ; Emulation modulation ; Institution design
Choosing the criteria for choosing : The need for indirect normativity ; Coherent extrapolated volition : Some explications; Rationales for CEV ; Morality models ; Do what I mean ; Component list : Goal content; Decision theory; Epistemology; Ratification ; Getting close enough
The strategic picture : Science and technology strategy : Differential technological development; Preferred order of arrival; Rates of change and cognitive enhancement; Technology couplings; Second-guessing ; Pathways and enablers : Effects of hardware progress; Should whole brain emulation research be promoted?; The person-affecting perspective favors speed ; Collaboration : The race dynamic and its perils; On the benefits of collaboration; Working together
Crunch time : Philosophy with a deadline ; What is to be done? : Seeking the strategic light; Building good capacity; Particular measures ; Will the best in human nature please stand up