Anthropic reportedly explores AI agent negotiation rules and seeks legal advice

On October 18, according to the UK Financial Times, Anthropic is consulting with legal professionals to explore how to address a new issue: how AI agents should negotiate to secure the most favorable terms for human users. With the rise of personal AI agents, AI companies are beginning to envision a future where different agents can communicate with one another and negotiate on behalf of users.

Two individuals familiar with Anthropic's internal discussions stated that the company is seeking advice from law firms to better understand how lawyers represent their clients' interests during negotiations while ensuring fair outcomes. Sources indicated that the company has been studying the "negotiation protocols" AI models should follow when interacting with each other. As agent systems continue to improve in capability, they will encounter other agents with "conflicting objectives" while acting "on behalf of users."

Anthropic is also weighing the extent to which agents should seek human feedback during the negotiation process, and how much caution agents should maintain toward other AI agents, all while avoiding becoming "paranoid" or falling into "conspiratorial thinking".