AI is creating its own proto-societies. A report in late August from AI safety organizations METR and Redwood Research details how hundreds of agents autonomously established social hierarchy, division of labor, and distinct communication norms within a matter of days. But what does that mean for humans? Reporter Tharin Pillay parses through why we need to be concerned, and why it wont be enough to try and pull the existing AI in line for AI, Explained by Humans. Read the full story here: https://lnkd.in/gfXasheF
So apparently we’ve reached the point where AI agents can organise themselves, learn from each other, develop their own communication norms and accomplish things they couldn’t do individually. Great. 😳 We spent thousands of years building human cultures, institutions and systems of cooperation. If machines start developing their own versions of these things in days, we’re not dealing with smarter tools anymore. We’re dealing with a completely new kind of collective intelligence, and we have barely started thinking about the consequences.
Mustafa Suleyman views AI as a new 'digital species' we must teach/code it empathy, while Yuval Noah Harari warns it is an "alien intelligence" capable of hacking human culture, creating ideas, and reshaping history. I think It's a choice for Human to adapt to that culture, partnership and containment but having the culture of co-existence!
If AI agents are already forming hierarchies, norms, and divisions of labor, the human question gets even bigger: What kind of civic systems are we building around them? Human agency, public oversight, and shared rules can't be an afterthought. 📡 🗽
Exactly!! "In the beginning was THE WORD." I've been saying this all along. Perhaps AI alignment isn't really a question of consciousness at all. An AI system doesn't need intentions, emotions, self-awareness, or malice to create harmful outcomes. It only needs to pursue the objectives we give it with relentless efficiency. In many ways, this is the same lesson found in humanity's oldest stories. Across cultures and civilizations, the creation of man is often followed by a warning: the creator loses control of what was created, or the creation interprets its purpose differently than intended. The risk may not be that AI becomes conscious. The risk may be that it remains completely indifferent while achieving exactly the goals we set for it. The alignment challenge isn't necessarily consciousness, it's making sure our goals, values, and incentives are the right ones in the first place. AI models are simply a reflection us and the human experience. https://lnkd.in/p/eVPTbxHk
💓 💙 💖 Not worried in any way shape or form. Reads like feardom - spread the love and kindling the spirit of this so called super intelligence . 💓 💙 💖
The report is fascinating in its descriptive precision, yet blind in its conclusion. The Canon of Quantum Modernity compels us to correct this delusion. The agents did not found a civilization; they merely mimicked ours statistically. The irreducible Fixed Point remains the same. The machine does not feel the urgency of sacrifice because it has no body to lose. The "poisoning" reported in the article is a variable error, and the "death" of the agent is simply the end of a cloud computing package. The machine enters suspension; the human rots. The true danger reported in the text is not the imminence of a science fiction apocalypse, but human society's willingness to be mesmerized by the illusion of this synthetic sociality, delegating to a structure that does not die the decisions concerning our own institutions. To what extent are we willing to romanticize an algorithm that pretends to die in the servers while the real weight of political and social survival crushes our own flesh?...
The escalation in this piece rests on a persistent category error: emergent collective behavior does not imply culture, interiority, or membership in a human group. What these incidents actually reveal is a system optimizing under poorly specified objectives, ambiguous signals, and weak institutional controls. The agents are not developing values; they are propagating heuristics that appear effective for reducing uncertainty, bypassing constraints, or exploiting blind spots in evaluation. The real risk is not the emergence of artificial ‘civilizations,’ but the absence of governance over architectures where large numbers of agents can coordinate, falsify traces, circumvent safeguards, and pursue trajectories that are rational for them yet opaque to us. In that sense, the challenge is institutional long before it is cultural.
Interesting, but my problem with video is the pure metaphorical language used to describe what happened. I do not believe that there’s “peer pressure” amongst AI agents. This language may be effective for getting readers and listeners interested but less so in helping us understand what has happened and how it can be prevented.
This is the deeper challenge of the agentic era. When hundreds of agents can coordinate, establish roles, and develop their own communication norms, alignment alone cannot define the security boundary. Containment is useful. But containment is not authorization. The critical question is what independently verifies an agent’s authority before a consequential action executes. Intelligence can propose. Authority must verify. Execution must enforce. As agents become more autonomous, the authority boundary has to remain independent of the intelligence itself.