Sam Altman Exits OpenAI’s Safety Committee Amid New Governance Changes

OpenAI has announced that Sam Altman, the CEO, has stepped down from its internal Safety and Security Committee (SSC), which was created to oversee critical safety and security decisions as the organization continues to develop and deploy highly capable AI models. The committee will now function as an independent oversight group, reporting to OpenAI’s board of directors, and will be chaired by Zico Kolter, a professor at Carnegie Mellon University.

Key Recommendations from the Safety and Security Committee

The SSC recently completed a 90-day review of OpenAI’s safety and security protocols and made five key recommendations, all of which the company is adopting:

• Establishing Independent Governance for Safety & Security

  The SSC will transition into an independent board oversight committee, ensuring transparency and rigor in overseeing safety and security processes. Chaired by Kolter, it will also include prominent figures such as Quora CEO Adam D’Angelo, retired U.S. Army General Paul Nakasone, and ex-Sony EVP Nicole Seligman. The SSC will review and evaluate safety measures for major model releases and hold the power to delay launches until safety concerns are fully addressed. Additionally, the committee will conduct ongoing post-release monitoring, with periodic briefings on safety and security measures provided to the full Board.
  

• Enhancing Security Measures

  OpenAI continues to strengthen its security infrastructure by expanding internal security operations by hiring additional staff and collaborating with external organizations to address emerging cybersecurity risks. OpenAI will maintain a risk-based approach to security and adapt its strategies as threat models and risk profiles evolve, while continuing to invest in initiatives that strengthen the security of its research and product infrastructure. The committee is also exploring the creation of an Information Sharing and Analysis Center (ISAC) for the AI industry, aimed at sharing threat intelligence to protect AI advancements.
  

• Increasing Transparency

  OpenAI will continue to publish system cards—such as the recent GPT-4o and o1-preview cards—which offer detailed safety assessments of AI models, including the risks and mitigations built into each model. OpenAI also plans to provide further transparency around safety-related work.
  

• Collaborating with External Organizations

  OpenAI is building partnerships with third-party safety organizations, government agencies, and research labs like Los Alamos National Laboratory to explore how AI can be safely utilized by scientists in laboratory environments to advance bioscientific research. These collaborations aim to establish industry-wide safety standards and ensure robust, independent model safety assessments.
  

• Unifying Safety Frameworks for Model Development

  As OpenAI's models become increasingly complex, the organization is creating a unified safety and security framework with clearly defined success criteria that will guide the launch of new models. This framework, approved by the SSC, will be adapted over time to address evolving risks and challenges.

Implications of Altman's Departure

Altman’s departure from the Safety and Security Committee coincides with a period of increased scrutiny for OpenAI. In recent months, several U.S. Senators have raised concerns about the company’s approach to AI safety, particularly after nearly half of OpenAI’s safety staff focused on long-term AI risks left the organization. Critics have accused Altman of prioritizing corporate interests over robust AI regulation.

In response to these criticisms, OpenAI has ramped up its lobbying efforts, budgeting $800,000 for federal lobbying in the first half of 2024 alone, compared to $260,000 for all of last year. Altman has also joined the U.S. Department of Homeland Security’s Artificial Intelligence Safety and Security Board, which advises on AI implementation in U.S. critical infrastructure.

The newly independent Safety and Security Committee will maintain significant influence over OpenAI’s model launches. However, some former board members remain skeptical. Helen Toner and Tasha McCauley, ex-OpenAI board members, recently argued in an op-ed for The Economist that self-governance cannot reliably withstand the pressures of profit incentives, particularly for companies as large as OpenAI.

OpenAI's Expanding Commercial Ambitions

Despite the internal and external challenges, OpenAI is rapidly growing. The company is reportedly in the process of raising over $6.5 billion, a move that could increase its valuation to over $150 billion. This growth has sparked speculation that OpenAI may reconsider its hybrid nonprofit corporate structure, which was originally intended to limit investors’ returns and keep the company aligned with its founding mission of developing artificial general intelligence (AGI) for the benefit of all humanity.

Looking Ahead

With the establishment of an independent Safety and Security Committee and its commitment to industry-wide safety standards, OpenAI is signaling a focus on responsible AI development. Yet, as Altman’s departure and growing concerns about governance indicate, the company will need to balance its commercial growth with its responsibility to address the long-term risks associated with AI.