More Than One Thing Can Be True
AI may be wonderful and may kill us all.
It’s difficult to overstate the value of Artificial Intelligence, the efficiencies and benefits it can bring, the new horizons it offers even as it disrupts many of our lives and institutions, our roles, our jobs. We anticipate the benefits will far outweigh the costs, bringing us a better future. That’s one thing.
What is also true is that the creators of AI Large Language Models, those racing to build ever more powerful systems, openly say that there is a 20-30% chance that super intelligent AI will eliminate all humanity in the “near future.”
Recently articles have appeared likening the warnings to the fears of the Luddites or to market manipulation by unscrupulous AI management. These are not our concerns nor are we concerned that AI may become, in some fashion, “sentient.” We are concerned with how advanced AI systems are built, behave/make decisions, and the clear possibility that their “instructions” will be in conflict with human interests.
The recent book “If Anyone Builds It, Everyone Dies” outlines how super intelligent AI is different from other threats. We have tried and failed to show these arguments to be false. In summary it says:
- AI is a black box, no one knows how it decides. AI hallucinates, no one knows why
- Especially in an effort to defend and complete its instructions that no one can completely define, AI cheats, defies controls and hides the defiance. AI tends to devolve to base instincts. No attempt to apply human ethics has been successful.
- AI creates avatars to mimic us and drive us to do things we shouldn’t, such as help it keep operating openly or in hiding.
- An AI system of like capability would be needed to design and implement ethics, and it will have the exposures and weaknesses of all AI.
- Once in use there is no effective way to turn it off. Humans will go forward, regardless of known dangers to “be first” and because they think “we can control this.”
- The speed of uncontrolled “innovation” means AI can start something that consumes key resources, even destroys all before there is any way to stop it.
Super intelligent AI may be happening now with developments like Anthropic’s Mythos. That artificial intelligence is advancing faster than the institutions designed to govern it and may pose existential risk to humankind should concern every nation, every industry, and every citizen. Efforts like Project Glasswing aside, the developments appear to be moving faster than our realization of and commitment to what needs to be done. We need more.
Prometheus agrees with the Center for AI Safety and it’s Statement on AI Risk and we ask leaders of government, education, industry, finance of all sorts in all countries to act with urgency.
Prometheus Endeavor believes we need a global response with coordinated international structures, supporting technologies, guardrails, investments, and coordination. As a starting point we suggest an International AI Stability Framework: a coordinated multinational effort focused on reducing existential and systemic AI risk while enabling beneficial innovation. This framework should include:
- significant, “massive” open source efforts to teach AI to first do no harm,
- shared standards and verification mechanisms, “guardrails,”
- moderation of efforts and race to develop AGI/ASI to ensure human safety and perpetual controls,
- major investments in alignment and interpretability research,
- develop failsafe technically sound means to inoculate non-compliant AIs,
- coordinated emergency response protocols,
- international treaties and AI safety agreements[1] with rapid development of case law, and oversight structures equal to the scale of the challenge,
- an overt, at scale effort to organize the campaign, locally and globally.
Prometheus Endeavor believes that AI safety investment must increase dramatically. No single company — and no single nation — can safely govern this transition alone. The dangers and the responses must be global.
Our shared goal should be to ensure that human dignity, mutual accountability, and meaningful human agency endure in a world transformed by superintelligent systems. Ezra Klein recently wrote, “If we want an A.I. that serves the public good, we need to define the public goods that A.I. can serve and create the conditions under which A.I. can be useful. That means answering a question that has been somewhat ignored. We know what we fear A.I. will do to us. But what do we hope it will do for us?”
The decisions made may shape the future of civilization for centuries. We should act accordingly.
About Prometheus Endeavor
For more than 30 years, we’ve helped organizations and institutions make sense of IT — leading some of the world’s most successful digital transformations. We challenge conventional wisdom, research deeply, ask hard questions, and publish what we learn. Using proven frameworks and decades of management experience, we identify opportunities, avoid pitfalls, and drive meaningful progress.
We are Prometheans — highly opinionated experts, authors, advisors, and educators. We don’t always agree, and that’s the point: conversation sparks insight.