Skip to content

Former Anthropic employee: Amodei once discussed that the company could potentially be more powerful than the government in the future, and superintelligent AI is like the "One Ring."

Nov 19, 19:20

Beating AI News Flash: Anthropic is about to go public, and many employees are expected to become overnight millionaires. But according to interviews by The New York Times, some people are currently more concerned about: if the company is the first to create superintelligence, exactly how much power will it gain?

The New York Times columnist Jasmine Sun has interviewed dozens of Anthropic founders, employees, and former employees over the past two years. One anonymous former employee revealed that CEO Dario Amodei has talked about how Anthropic could potentially become more powerful than governments in the future, and therefore needs to establish its own relationships with various countries.

Another former employee used the "One Ring" from The Lord of the Rings as a metaphor for superintelligence. Whoever creates it first may gain control of the most dangerous cyber weapons, methods to cure various diseases, and an tireless AI army. Once such immense power is concentrated in the hands of a few, whether those who wield it are trustworthy becomes crucial.

Amodei's attitude toward this race has also changed markedly. In 2022, he publicly stated that Anthropic should not jump the gun, nor should it train models far larger than those of its peers. That year, the company even delayed releasing Claude to the public due to safety concerns, after which OpenAI seized the market initiative with ChatGPT.

Now, Amodei believes the AI race can no longer be stopped. If Anthropic does not develop the most advanced models first, other companies will continue to push forward. Only by staying ahead can Anthropic have enough influence to push the entire industry toward safer practices.

This sense of mission has also permeated Anthropic's hiring process. The company has a famous interview question: if slowing down research for AI safety caused your stock value to go to zero, would you still be willing to work here? The research team will even reject candidates with outstanding technical ability who do not agree with the company's philosophy.

As the company expands rapidly, internal divisions have begun to emerge. Some early employees worry that newly joined sales and marketing staff are only drawn by high pay and stock, and may not truly believe in the company's safety mission. Some employees also advocate immediately stopping the training of stronger AI, even if competitors such as OpenAI will not stop along with them, but this was rejected by Amodei.

Anthropic is about to obtain massive funding through an IPO to continue purchasing computing power and developing stronger models. At the same time, the company is also seeking an antitrust exemption from Congress so that in the future it can coordinate with other U.S. AI labs to slow down research and development.

Source