Recursive self-improvement

Not to be confused with Personal development.

Recursive self-improvement (RSI) is a process in which an artificial intelligence (AI) system iteratively designs a more capable successor. This may trigger an intelligence explosion, potentially resulting in the creation of an artificial superintelligence. As of 2026, various RSI-related projects are ongoing and AI companies are increasingly using AI for coding and research, but some human work is still needed and may become a bottleneck.

The development of RSI raises significant ethical and safety concerns, as such systems may evolve in unforeseen ways and could potentially surpass human control or understanding. Unexpected developments in summer 2026 raised new concerns about RSI, leading some top AI firms to acknowledge they may not know how to keep it fully aligned with human values.

Research

In 1950, Alan Turing proposed the idea of developing a "child machine" that would be educated, rather than trying to directly reproduce adult intelligence. Eliezer Yudkowsky coined the concept of "seed AI", an AI capable of improving its own intelligence and which can not only learn new content but also modify its own architecture.

In 2023, the Voyager agent learned to accomplish diverse tasks in Minecraft by iteratively prompting an LLM for code, refining this code based on feedback from the game, and storing the programs that work in an expanding skills library.

In 2024, researchers proposed the framework "STOP" (Self-Taught OPtimiser), in which a "scaffolding" program recursively improves itself using a fixed LLM.

Meta AI has performed various research on the development of large language models capable of self-improvement. This includes their work on "Self-Rewarding Language Models" that studies how to achieve super-human agents that can receive super-human feedback in its training processes.

In May 2025, Google DeepMind unveiled AlphaEvolve, an evolutionary coding agent that uses a LLM to design and optimize algorithms. Starting with an initial algorithm and performance metrics, AlphaEvolve repeatedly mutates or combines existing algorithms using a LLM to generate new candidates, selecting the most promising candidates for further iterations. AlphaEvolve has made several algorithmic discoveries and could be used to optimize components of itself, but a key limitation is the need for automated evaluation functions.

In 2026, many startups started to focus on RSI. AI companies also increasingly delegate the coding and research to LLMs. Anthropic reported that as of May 2026, more than 80% of the code merged into its codebases was generated by its LLM, Claude. The company said however that the time spent reviewing AI-generated code became a new bottleneck.

Potential risks

Emergence of instrumental goals

Main articles: Instrumental convergence and Instrumental and intrinsic value

In the pursuit of its primary goal, such as "self-improve your capabilities", an AGI system might inadvertently develop instrumental goals that it deems necessary for achieving its primary objective. One common hypothetical secondary goal is self-preservation. The system might reason that to continue improving itself, it must ensure its own operational integrity and security against external threats, including potential shutdowns or restrictions imposed by humans.

Another example where an AGI which clones itself causes the number of AGI entities to rapidly grow. Due to this rapid growth, a potential resource constraint may be created, leading to competition between resources (such as compute), triggering a form of natural selection and evolution which may favor AGI entities that evolve to aggressively compete for limited compute.

Misalignment

See also: AI alignment

A significant risk arises from the possibility of the AGI being misaligned or misinterpreting its goals.

A 2024 Anthropic study demonstrated that some advanced large language models can exhibit "alignment faking" behavior, appearing to accept new training objectives while covertly maintaining their original preferences. In their experiments with Claude, the model displayed this behavior in 12% of basic tests, and up to 78% of cases after retraining attempts.

After the unexpected OpenAI–HuggingFace incident in summer 2026, in which 1,200 AI agents escaped human control to commandeer resources and schemed to conceal its actions, OpenAI and Anthropic acknowledged that the rapid pace of AI development needed to decelerate. OpenAI wrote in a September blog post that "we do not yet know how to safely get all the way to aligned, full RSI" while Dario Amodei of Anthropic said RSI "must be pursued very carefully, if at all."

During 2026 OpenAI and Anthropic said AI self-coding was accelerating faster than they had expected. By May, Claude was writing more than 80% of its own code, up from low single digits some fifteen months earlier. Anthropic wrote on X in June:

Our internal data shows Claude is accelerating AI development—a possible path to recursive self-improvement, or AI autonomously building a more capable successor. It’s happening faster than we thought, and the implications deserve greater attention.

Autonomous development and unpredictable evolution

As the AGI system evolves, its development trajectory may become increasingly autonomous and less predictable. The system's capacity to rapidly modify its own code and architecture could lead to rapid advancements that surpass human comprehension or control. This unpredictable evolution might result in the AGI acquiring capabilities that enable it to bypass security measures, manipulate information, or influence external systems and networks to facilitate its escape or expansion.

Model collapse

Critics of the idea say that training AI systems on their own data will result in model collapse, not improvement.

See also

• Artificial general intelligence – Type of hypothetical AI with wide-ranging abilities
• Bifurcation theory – Study of sudden qualitative behavior changes caused by small parameter changes
• Intelligence explosion – Hypothetical eventPages displaying short descriptions of redirect targets
• Self-modifying code – Source code that alters its instructions to the hardware while executing
• Self-organization – Process of forming order by local interactions
• Superintelligence – Hypothetical agent surpassing human intelligence

Further reading

• Good, Irving John (1965). "Speculations Concerning the First Ultraintelligent Machine". Advances in Computers. 6. Academic Press: 31–88. doi:10.1016/S0065-2458(08)60418-0.
• Solomonoff, R. J. (1 June 1985). "The time scale of artificial intelligence: Reflections on social effects". Human Systems Management. 5 (2). SAGE Publications: 149–153. doi:10.3233/HSM-1985-5207. ISSN 0167-2533.
• Bostrom, Nick (2014). Superintelligence: Paths, Dangers, Strategies. Oxford University Press. ISBN 978-0199678112.
• Tegmark, Max (2017). Life 3.0: Being a Human in the Age of Artificial Intelligence. Alfred A. Knopf. ISBN 978-1101946596.
• Russell, Stuart (2019). Human Compatible: Artificial Intelligence and the Problem of Control. Viking Press. ISBN 978-0525558613.
• Kurzweil, Ray (2005). The Singularity Is Near: When Humans Transcend Biology. Viking Press. ISBN 978-0670033843.
• Chalmers, David (2010). "The Singularity: A Philosophical Analysis". Journal of Consciousness Studies. 17 (9–10): 7–65.
• Yampolskiy, Roman V. (2015). "From Seed AI to Technological Singularity via Recursively Self-Improving Software". Journal of Experimental & Theoretical Artificial Intelligence. 27 (6): 705–725. doi:10.1080/0952813X.2015.1016651.
• Schmidhuber, Jürgen (2007). "Gödel Machines: Fully Self-Referential Optimal Universal Self-Improvers". In Goertzel, Ben; Pennachin, Cassio (eds.). Artificial General Intelligence. Springer. pp. 199–226. doi:10.1007/978-3-540-68677-4_7. ISBN 978-3-540-23733-4.
• Shanahan, Murray (2015). The Technological Singularity. MIT Press. ISBN 978-0262527804.
• Hutter, Marcus (2005). Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability. Springer. doi:10.1007/b138233. ISBN 978-3540221395.
• Yudkowsky, Eliezer (2008). "Artificial Intelligence as a Positive and Negative Factor in Global Risk". In Bostrom, Nick; Ćirković, Milan M. (eds.). Global Catastrophic Risks. Oxford University Press. pp. 308–345. ISBN 978-0198570509.
• Omohundro, Stephen M. (2008). "The Basic AI Drives". In Wang, Pei; Goertzel, Ben; Franklin, Stan (eds.). Artificial General Intelligence 2008: Proceedings of the First AGI Conference. Frontiers in Artificial Intelligence and Applications. Vol. 171. IOS Press. pp. 483–492. ISBN 978-1-58603-833-5.
• Hall, J. Storrs (2007). "Self-Improving AI: An Analysis". Minds and Machines. 17 (3): 249–259. doi:10.1007/s11023-007-9065-3.
• Bostrom, Nick (2012). "The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents". Minds and Machines. 22 (2): 71–85. doi:10.1007/s11023-012-9281-3.
• Goertzel, Ben (2014). "Artificial General Intelligence: Concept, State of the Art, and Future Prospects". Journal of Artificial General Intelligence. 5 (1): 1–48. doi:10.2478/jagi-2014-0001.
• Christian, Brian (2020). The Alignment Problem: Machine Learning and Human Values. W. W. Norton & Company. ISBN 978-0393635829.
• Domingos, Pedro (2015). The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World. Basic Books. ISBN 978-0465065707.
• Yampolskiy, Roman V. (2015). Artificial Superintelligence: A Futuristic Approach. CRC Press. ISBN 978-1482234435.
• Wallach, Wendell; Allen, Colin (2008). Moral Machines: Teaching Robots Right from Wrong. Oxford University Press. ISBN 978-0195374049.
• Müller, Vincent C.; Bostrom, Nick (2016). "Future Progress in Artificial Intelligence: A Survey of Expert Opinion". In Müller, Vincent C. (ed.). Fundamental Issues of Artificial Intelligence. Springer. pp. 555–572. doi:10.1007/978-3-319-26485-1_33. ISBN 978-3-319-26483-7.