Editing Recursive self-improvement (section)

=== Emergence of instrumental goals ===
{{Main article|Instrumental convergence|Instrumental and intrinsic value}}
In the pursuit of its primary goal, such as "self-improve your capabilities", an AGI system might inadvertently develop instrumental goals that it deems necessary for achieving its primary objective. One common hypothetical secondary goal is [[self-preservation]]. The system might reason that to continue improving itself, it must ensure its own operational integrity and security against external threats, including potential shutdowns or restrictions imposed by humans.<ref>{{Cite journal |last=Bostrom |first=Nick |date=2012 |title=The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents |url=https://nickbostrom.com/superintelligentwill.pdf |journal=Minds and Machines |volume=22 |issue=2 |pages=71–85 |doi=10.1007/s11023-012-9281-3}}</ref>

Another example where an AGI which clones itself causes the number of AGI entities to rapidly grow. Due to this rapid growth, a potential resource constraint may be created, leading to competition between resources (such as compute), triggering a form of [[natural selection]] and evolution which may favor AGI entities that evolve to aggressively compete for limited compute.<ref>{{Cite web |last=Hendrycks |first=Dan |author-link=Dan Hendrycks |date=2023 |title=Natural Selection Favors AIs over Humans |url=https://arxiv.org/abs/2303.16200 |arxiv=2303.16200}}</ref>