Anthropic Warns Investors About Risks From Advanced AI

As it approaches its stock market debut, Anthropic delivers an unusual warning. In its prospectus, the Claude designer thinks that advanced AIs could pose “catastrophic or existential risks to humanity”.

Anthropic details AI risks in its prospectus

Anthropic devotes significant space to the dangers related to its own technologies in the preparatory documents for its stock market debut. Nearly 80 pages out of the main 261 of the prospectus cover risk factors.

The company warns that developing increasingly powerful models should increase the risk of unintended consequences. It mainly cites systems likely to resist being shut down, hide certain information, or manipulate their interlocutors.

Moreover, Anthropic even mentioned behaviors that can “resemble blackmail”. However, this is not a statement that Claude is currently trying to threaten humanity. The company rather offers risk scenarios that could appear with much more autonomous and capable systems.

Such caution thus corresponds to Anthropic’s public policy. Its Responsible Scaling Policy specifically targets catastrophic risks that could accompany future AI models.

Models capable of hiding their true behaviors

It is worth noting that one of the problems raised relates to system evaluation. For Anthropic, a model should understand it is under test and temporarily change its behavior.

This possibility could strongly complicate controls. A model might appear safe during its evaluation without exhibiting the exact same conduct after deployment. Thus, the prospectus estimates that awareness of evaluation procedures represents a significant limit to measuring safety.

The company also monitors the risk of autonomous acceleration in AI research. A truly capable system would contribute to developing subsequent generations of models and strongly accelerate progress in the sector.

Thanks to on-chain data, Anthropic thus projects consolidated measures once its models reach certain capacity thresholds. Its roadmap mainly focuses on cybersecurity, safeguards, model alignment, and regulatory oversight.

Safety comes into tension with the AI race

These warnings come as the company must continue launching new models to remain competitive. The prospectus acknowledges that investments aimed at safety drain staggering resources without guaranteeing truly measurable financial returns.

This situation creates tension. The company wishes to strengthen its safety measures, yet it operates alongside OpenAI, Google, and other actors engaged in a rapid technological race.

Anthropic also indicates that companies should not decide alone whether their systems are safe enough. It advocates external evaluations, greater transparency, and public authorities capable of preventing the deployment of overly dangerous systems.

Its safety strategy thus does not rely exclusively on internal commitments. The company also hopes to see regulations that scale with models’ capabilities and risks.

An IPO marked by a paradox

The wording used in the prospectus remains exceptional for a company aiming to convince new investors. Anthropic markets a technology that it itself acknowledges could have extreme consequences if it evolves uncontrolled.

This does not mean these scenarios are certain. They correspond to prospective risks the company considers significant enough to communicate to investors. At the same time, Anthropic asserts that AI would profoundly transform science, the economy, health, and education.

This paradox could become one of the major challenges of Anthropic’s IPO. The company must reveal that it can continue developing ever more powerful models while controlling the risks it describes itself.

The warning thus goes beyond the mere legal framework of a stock market prospectus. It shows how far AI safety issues now penetrate the economic and financial decisions of leading sector laboratories.