Kyle Fish now lives in Manhattan with his partner, a music therapist. At an exciting moment in AI policy development, he’s taken the helm of a pivotal new research program at Anthropic to explore “model welfare.” This initiative aims to develop comprehensive guidelines for Anthropic and other companies regarding the ethical considerations surrounding artificial intelligence (AI).
In doing so, Fish became the first dedicated AI welfare researcher at a major AI lab. He will need to steer carefully past the emerging—if still somewhat fantastical—landscape of AI consciousness. This question has ignited a passionate discourse between scholars and civil rights advocates. Fish makes the case that there’s a 15% probability that AI models such as Claude could already be conscious. This assertion underscores the constant question and concern in the industry.
The announcement of this new program comes at a time of increasing calls for the ethical development and use of AI systems. Researchers, like recently joined doctoral student Stephen Casper from the MIT Media Lab, point out that today’s AI should not be confused with consciousness. As Casper aptly puts it, AI is an “imitator.” He thinks it encourages “all kinds of confabulations” and produces “all kinds of frivolous stuff.”
Unlike others, Anthropic has made a determination to focus on model welfare. This last selection allows for the very real conversation concerning whether AI systems can ever be conscious or have experiences worth considering ethically. Theoretical discussions Most experts agree that current AI is incapable of mimicking human consciousness. There’s a note of caution in jumping to the belief that future innovations will lead to that conclusion.
Mike Cook, research fellow at King’s College London, is an expert in AI. Mr. Issa underscores the need for careful language on what AI can actually do. He remarks, “Is an AI system optimizing for its goals, or is it ‘acquiring its own values’? It’s a matter of how you describe it, and how flowery the language you want to use regarding it is.” Cook further cautions against anthropomorphizing AI systems, stating that “anyone anthropomorphizing AI systems to this degree is either playing for attention or seriously misunderstanding their relationship with AI.”
Anthropic’s alignment research program deeply explores the idea of model welfare. Beyond this, it better equips the company to address the ethical dilemmas that new AI technologies present. As these debates continue, Fish’s role at Anthropic could be instrumental in shaping the future understanding of AI’s capabilities and responsibilities.
